Schema and Data Translation

Preview:

DESCRIPTION

Schema and Data Translation. Paolo Atzeni Università Roma Tre Based on Tutorial at ICDE 2006 Paper in EDBT 2006 (with P. Cappellari and P. Bernstein) Demo in Sigmod 2007 (with P. Cappellari and G. Gianforme) ADBIS --- Varna , October 2, 2007. Outline. Introduction Model management - PowerPoint PPT Presentation

Citation preview

Schema and Data Translation

Paolo AtzeniUniversità Roma Tre

Based onTutorial at ICDE 2006

Paper in EDBT 2006 (with P. Cappellari and P. Bernstein)Demo in Sigmod 2007 (with P. Cappellari and G. Gianforme)

ADBIS --- Varna, October 2, 2007

P. Atzeni Varna - October 2, 2007 2

Outline

• Introduction• Model management• Schema and data translation: the problem• A metamodel based approach

P. Atzeni Varna - October 2, 2007 3

A ten-year goal for database research

• The “Asilomar report”(Bernstein et al. Sigmod Record 1999 www.acm.org/sigmod):– The information utility:

make it easy for everyone to store, organize, access, and analyze the majority of human information online

P. Atzeni Varna - October 2, 2007 4

A general framework: cooperation

• "The capacity of a system to interact (effectively) with other systems, possibly managed by different organizations"

• Forms of cooperation– Process-centered cooperation:

• the systems offer services one another, by exchanging messages, or by triggering activities, without making remote data explicitly visible

– Data-centered cooperation: • the systems offer data one another; data is distributed,

heterogeneous and autonomous

P. Atzeni Varna - October 2, 2007 5

Databases in the Internet era

• Databases before the Internet– An internal infrastructure, a precious resource, but usually

hidden, with some controlled cooperation• Internet changes the requirements

– More users (not only humans), more diverse cooperating systems (distributed, heterogeneous, autonomous), more types of data

• "Future" Internet changes more– New devices (embedded everywhere), even more users

(many “per person”), real mobility, need for personalization and adaptation

P. Atzeni Varna - October 2, 2007 6

The most studied form of data-centered cooperation: integration

• We are interested in data-centered cooperation, often referred to as integration

“The unification of related, heterogeneous data from disparate sources, for example, to enable collaboration” (Hammer & Stonebraker 2005)

• Some "paradigms" …

P. Atzeni Varna - October 2, 2007 7

Multidatabase

Global Manager

Local mgr

DB

Mediator

Local mgr

DB

Mediator

Local mgr

DB

Mediator

P. Atzeni Varna - October 2, 2007 8

Data Warehousing System

Mediator

Local manager

Mediator

Local manager

Mediator

Local manager

“Integrator”

DB DB DB

Data Warehouse

DW manager

P. Atzeni Varna - October 2, 2007 9

Intermediate solutions in practice

Mediator

Local manager

DB

Local manager

DB

Integrator

Mediator

Local manager

Mediator

Local manager

DB DB

DB

Local Manager

Application

P. Atzeni Varna - October 2, 2007 10

Peer-based architecturePeer mgr

Local mgr

DB

Local mgr

DB

Mediator

Mediator

Peer mgr

Local mgr

DB

Local mgr

DB

Mediator

Mediator

Peer mgr

Local mgr

DB

Local mgr

DB

Mediator

Mediator

P. Atzeni Varna - October 2, 2007 11

Data is not just in databases

• Mail messages• Web pages• Spreadsheets• Textual documents• Palmtop devices, mobile phones• Multimedia annotations (e.g., in digital photos)• XML documents

P. Atzeni Varna - October 2, 2007 12

The same data in the same form?

• Adaptivity:– Personalization: content adapted to the user

• upon system's decision• upon user's request

– Customization: structure adapted to the user• according to the user's role• upon user's request

– Context dependence• User, Device, Network, Place, Time, Rate

P. Atzeni Varna - October 2, 2007 13

Summarizing: a general need

• We have data at various places, and data has to be– exchanged– replicated – migrated– integrated – adapted

P. Atzeni Varna - October 2, 2007 14

A major difficulty

• Heterogeneity– System level– Model level– Structural (different structure for similar data)– Semantic (different meaning for the same structure)

• Many efforts, but current techniques are mostly manual and ad hoc

P. Atzeni Varna - October 2, 2007 15

A direction for the solutions

• Be general! – ad hoc solution could work in-the-small, but they

• are repetitive and time consuming • do not scale• are not maintainable

• Historical notes:– W. C. McGee: Generalization: Key to Successful Electronic

Data Processing. J. ACM 1959• Indeed, databases are the result of generalization applied to

secondary storage management!

P. Atzeni Varna - October 2, 2007 16

Generality requires …

• … high-level descriptions of problems within the family of interest:– Metadata:

• “data about data”• (formal or informal) description of structures and

meaning

• General solutions leverage on metadata (and then operate on data as a consequence)

P. Atzeni Varna - October 2, 2007 17

Outline

• Introduction and motivation• Model management• Schema and data translation: the problem• A metamodel based approach

P. Atzeni Varna - October 2, 2007 18

A wider perspective

• (Generic) Model Management:– A proposal by Bernstein et al (2000 +)– Includes a set of operators on

• schemas and • mappings between schemas

P. Atzeni Varna - October 2, 2007 19

Terminology: a warning

Model Mgmt people Traditional DB people

Meta-metamodel Metamodel

Metamodel Model

Model Schema

P. Atzeni Varna - October 2, 2007 20

Schemas and mappings

• A simplified approach:– Schema:

• a set of elements, related in some way to one another– Mapping:

• a set of correspondences (pair of elements) or• its reification, a third schema related to the other two via

two sets of correspondences

P. Atzeni Varna - October 2, 2007 21

Model mgmt operators, a first set

• map = Match (S1, S2) • S3 = Merge (S1, S2, map)• S2 = Diff (S1, map) • and more

– map3 = Compose (map1, map2)

– S2 = Select (S1, pred)

– Apply (S, f)

– list = Enumerate (S)

– S2 = Copy (S1)

– …

P. Atzeni Varna - October 2, 2007 22

Match

• map = Match (S1, S2)– given

• two schemas S1, S2– returns

• a mapping between them• the “classical” initial step in data integration:

– find the common elements of two schemas and the correspondences between them

P. Atzeni Varna - October 2, 2007 23

Merge

• S3 = Merge (S1, S2, map)– given

• two schemas and a mapping between them– returns

• a third schema (and two mappings)• the “classical” second step in data integration:

– given the correspondences, find a way to obtain one schema out of two

P. Atzeni Varna - October 2, 2007 24

Diff

• S2 = Diff (S1, map) – given

• a schema and a mapping from it (to some other schema, not relevant)

– returns • a (sub-)schema, with the elements that do not participate

in the mapping

P. Atzeni Varna - October 2, 2007 25

DW2

Example

(Bernstein and Rahm, ER 2000)• A database (a “source”), a data warehouse and a mapping

between the two• we get a second source, with some similarity to the first one• and we want to update the DW

DB1 DW1

DB2

P. Atzeni Varna - October 2, 2007 26

DW2

Example, the "solution"

DB1 DW1

DB2

m1

m2m3

DB2’

m2 = Match(DB1,DB2)

m3= Compose(m2,m1)

DB2’=Diff(DB2,m3)

DW2’, m4 user defined

m5 = Match(DW1,DW2’)

DW2 = Merge(DW1,DW2’,m5)

DW2’m4

m5

P. Atzeni Varna - October 2, 2007 27

Magic does not exist

• Operators might require human intervention:– Match is the main case

• Scripts involving operators might require human intervention as well (or at least benefit from it):– a full implementation of each operator might not always

available– a mapping might require manual specification– incomparable alternatives might exist

P. Atzeni Varna - October 2, 2007 28

The “data level”

• The major operators have also an extended version that operates on data, and not only on schemas

• Especially apparent for– Merge

P. Atzeni Varna - October 2, 2007 29

We also have heterogeneity

• Round trip engineering (Bernstein, CIDR 2003)– A specification (for example ER or UML) and an

implementation (for example, relational)– then a change to the implementation: want to revise the

specification• We need a translation from the implementation model to the

specification one

S1

I1 I2

S2

P. Atzeni Varna - October 2, 2007 30

Model management with heterogeneity

• The previous operators have to be “model generic” (capable of working on different models)

• We need a “translation” operator– <S2, map12> = ModelGen (S1)

P. Atzeni Varna - October 2, 2007 31

ModelGen, an additional operator

• <S2, map12> = ModelGen (S1) – given

• a schema (in a model)– returns

• a schema (in a different data model) and a mapping between the two

• A “translation” from a model to another• I should call it “SchemaGen” …• We should better write

– <S2, map12> = ModelGen (S1,mod2)

P. Atzeni Varna - October 2, 2007 32

S2

Round trip engineering

S1

I1

m1

I2

m2 = Match (I1,I2)m3 = Compose (m1,m2)I2’= Diff(I2,m3)<S2’,m4 > = Modelgen(I2’)… Match, Merge

m2

m3

I2’

S2’

m4

P. Atzeni Varna - October 2, 2007 33

Outline

• Introduction• Model management• Schema and data translation: the problem• A metamodel based approach

P. Atzeni Varna - October 2, 2007 34

Schema and data translation, a long standing issue

• Schema translation:– given schema S1 in model M1 and model M2– find a schema S2 in M2 that “corresponds” to S1

• Schema and data translation:– given also a database D1 for S1– find also a database D2 for S2 that “contains the same data”

to D1

P. Atzeni Varna - October 2, 2007 35

Schema and data translation, a long standing issue

• Translations from a model to another have been studied since the 1970’s

• Whenever a new model is defined, techniques and tools to generate translations are studied

• However, proposals and solutions are usually model specific

P. Atzeni Varna - October 2, 2007 36

Model specific solutions

• Given an ER schema, find the suitable relational schema that “implements” it – the original paper (Chen 1976) contains the basics– further discussions by many (e.g. Markowitz and Shoshani

1989)– illustrated in every textbook

• Similarly with – any other conceptual model and any other logical one– XML and relational (or object)

P. Atzeni Varna - October 2, 2007 37

Another problem in the picture: data exchange

• Given a source S1 and a target schema S2 (in different models or even in the same one), find a translation, that is, a function that given a database D1 for S1 produces a database D2 for S2 that “correspond” to D1

• Often emphasized with reference to materialized solutions

P. Atzeni Varna - October 2, 2007 38

Integration

• Given two or more sources, build an integrated schema or database

P. Atzeni Varna - October 2, 2007 39

Schema translation

• Given a schema find another one with respect to some specific goal (another model, better quality, …)

P. Atzeni Varna - October 2, 2007 40

Data exchange

• Given a source and a target schema, find a transformation from the former to the latter

P. Atzeni Varna - October 2, 2007 41

Schema translation and data exchange

• Can be seen a complementary:– Data translation = schema translation + data exchange

• Given a source schema and database• Schema translation produces the target schema• Data exchange generates the target database

• In model management terms we could write– Schema translation:

• <S2, map12> = ModelGen (S1,mod2)– Data exchange:

• i2 = DataGen (S1,i1,S2,map12)

P. Atzeni Varna - October 2, 2007 42

Outline

• Introduction• Model management• Schema and data translation: the problem• A metamodel based approach

P. Atzeni Varna - October 2, 2007 43

 The problem

• ModelGen– given two data models M1 and M2, and a schema S1 of M1

(the source schema and model), • generate a schema S2 of M2 (the target schema and

model), corresponding to S1• and, for each database D1 over S1, generate an

equivalent database D2 over S2

P. Atzeni Varna - October 2, 2007 44

We have been doing this for a while

• Initial work more than ten years ago (Atzeni & Torlone, 1996)• Major novelty recently (Atzeni, Cappellari & Bernstein, 2006)

– translation of both schemas and data– data-level translations generated automatically, from

schema-level ones

P. Atzeni Varna - October 2, 2007 45

 A simple example

• An object relational database, to be translated in a relational one

• Source: the OR-model

• Target: the relational model

System managed ids

used as references

P. Atzeni Varna - October 2, 2007 46

 Example, 2

• Does the OR model allow for keys? • Assume EmpNo and Name are keys

P. Atzeni Varna - October 2, 2007 47

 Example, 3

• Does the OR model allow for keys? • Assume no keys are specified

P. Atzeni Varna - October 2, 2007 48

Many different models (and variations …)

Binary ER w/o gen

N-ary ER w/ gen

Binary ERw/ gen

Relational

Bin ER w/ gen w/o attr on rel

OO w/ gen

N-ary ER w/o gen

Bin ER w/o gen w/o attr on rel

Bin ER w/o gen w/o M:N rel

Bin ER w/ gen w/o M:N rel

OO w/o gen…

OR

XSD

P. Atzeni Varna - October 2, 2007 49

Heterogeneity

• We need to handle artifacts and data in various models– Data are defined wrt to schemas– Schemas are defined wrt to models – How models can be defined?

Models

Schemas

Data

P. Atzeni Varna - October 2, 2007 50

A metamodel approach

• The constructs in the various models are rather similar:

– can be classified into a few categories (Hull & King 1986):

• Lexical: set of printable values (domain)

• Abstract (entity, class, …)

• Aggregation: a construction based on (subsets of) cartesian products (relationship, table)

• Function (attribute, property)

• Hierarchies

• …

• We can fix a set of metaconstructs (each with variants):

– lexical, abstract, aggregation, function, ...

– the set can be extended if needed, but this will not be frequent

• A model is defined in terms of the metaconstructs it uses

P. Atzeni Varna - October 2, 2007 51

 The metamodel approach, example

• The ER model:– Abstract (called Entity)– Function from Abstract to Lexical (Attribute)– Aggregation of abstracts (Relationship) – …

• The OR model:– Abstract (Table with ID)– Function from Abstract to Lexical (value-based Attribute)– Function from Abstract to Abstract (reference Attribute)– Aggregation of lexicals (value-based Table)– Component of Aggregation of Lexicals (Column)– …

P. Atzeni Varna - October 2, 2007 52

 The supermodel

• A model that includes all the meta-constructs (in their most general forms)– Each model is subsumed by the supermodel (modulo

construct renaming) – Each schema for any model is also a schema for the

supermodel (modulo construct renaming)

P. Atzeni Varna - October 2, 2007 53

The metamodel approach, translations

• The constructs in the various models are rather similar: – can be classified into a few categories (“metaconstructs'')– translations can be defined on metaconstructs,

• and there are “standard”, accepted ways to deal with translations of metaconstructs

• they can be performed within the supermodel– each translation from the supermodel SM to a target model

M is also a translation from any other model to M:• given n models, we need n translations, not n2 

P. Atzeni Varna - October 2, 2007 54

Generic translation environment

1. Copy

2. Translation

3. Copy

Supermodel

Source model

Target model

Translation:composition 1,2 & 3

P. Atzeni Varna - October 2, 2007 55

Translations within the supermodel

• We still have too many models:– Just within simple ER model versions, we have 4 or 5

constructs, and each has several independent features which give rise to variants

• for example, relationships can be– binary or N-ary– with all possible cardinalities or without many-to-

many– with or without the possibility of specifying optionality– with or without attributes– …

– Combining all these, we get hundreds of models!– The management of a specific translation for each model

would be hopeless

P. Atzeni Varna - October 2, 2007 56

Translations, the approach

• Elementary translation steps to be combined• Each translation step handles a supermodel construct (or a

feature thereof) "to be eliminated" or "transformed"• A translation is the concatenation of elementary translation

steps

P. Atzeni Varna - October 2, 2007 57

A complex translation, example(0,N) (0,N)

• Eliminate N-ary relationships • Eliminate attributes from relationships • Eliminate many-to-many relationships• Replace relationships with references• Eliminate generalizations

P. Atzeni Varna - October 2, 2007 58

Complex translations

Binary ER w/o gen

N-ary ER w/ gen

Binary ERw/ gen

Relational

Bin ER w/ gen w/o attr on rel

OO w/ gen

N-ary ER w/o gen

Bin ER w/o gen w/o attr on rel

Bin ER w/o gen w/o M:N rel

Elim. N-ary relationships Elim. Relationship attr.sElim. M:N relationshipsReplace relationships with

referencesElim OO generalizations Elim ER generalizations

Bin ER w/ gen w/o M:N rel

OO w/o gen…

P. Atzeni Varna - October 2, 2007 59

Translations

• Basic translations are written in a variant of Datalog, with OID invention– We specify them at the schema level– The tool "translates them down" to the data level– Some completion or tuning may be needed

P. Atzeni Varna - October 2, 2007 60

A Multi-Level Dictionary

• Handles models, schemas and data• Has both a model specific and a model independent component

P. Atzeni Varna - October 2, 2007 61

Multi-Level Dictionary

model

schema

modelgeneric

modelspecific

des

crip

tion

modelindependence

Supermodel description

(mSM)

Model descriptions

(mM)

Supermodel schemas

(SM)

Model specific schemas

(M)

Supermodel instances

(i-SM)

Model specific instances

(i-M)data

P. Atzeni Varna - October 2, 2007 62

The supermodel description

MSM-Construct

OID Name IsLex

1 AggregationOfLexicals F

2 ComponentOfAggrOfLex T

3 Abstract F

4 AttributeOfAbstract T

5 BinaryAggregationOfAbstracts F

MSM-Property

OID Name Constr. Type

11 Name 1 String

12 Name 2 String

13 IsKey 2 Boolean

14 IsNullable 2 Boolean

15 Type 2 String

16 Name 3 String

17 Name 4 String

18 IsIdentifier 4 Boolean

19 IsNullable 4 Boolean

20 Type 4 String

21 IsFunct1 5 Boolean

22 IsOptional1 5 Boolean

23 Role1 5 String

24 IsFunct2 5 Boolean

25 IsOptional2 5 Boolean

26 Role2 5 String

MSM-Reference

OID Name Construct Target

30 Aggregation 2 1

31 Abstract 4 3

32 Abstract1 5 3

33 Abstract2 5 3

mSMmM

SMM

i-SMi-M

P. Atzeni Varna - October 2, 2007 63

Model descriptions

MSM-Construct

OID Name IsLex

1 AggregationOfLexicals F

2 ComponentOfAggrOfLex T

3 Abstract F

4 AttributeOfAbstract T

5 BinaryAggregationOfAbstracts F

MM-Construct

OID Model MSM-Constr Name

1 2 3 ER_Entity

2 2 4 ER_Attribute

3 2 5 ER_Relationship

4 1 1 Rel_Table

5 1 2 Rel_Column

6 3 3 OO_Class

MM-Model

OID Name

1 Relational

2 Entity-Relationship

3 Object

MSM-Property

OID Name Construct Type

… … … …

MSM-Reference

OID Name Construct …

… … … …

MM-Property

… … … …

MM-Reference

… … … …

mSMmM

SMM

i-SMi-M

P. Atzeni Varna - October 2, 2007 64

Schemas in a model

ER-Entity

OID Schema Name

301 1 Employees

302 1 Departments

201 3 Clerks

202 3 Offices

ER-AttributeOfEntity

OID Schema Name isIdent isNullable Type AbstrOID

401 1 EmpNo T F Int 301

402 1 Name F F Text 301

404 1 Name T F Char 302

405 1 Address F F Text 302

501 3 Code T F Int 201

… … … … … … …

SM-Construct

OID Name IsLex

… … …

3 ER-Entity F

4 ER-AttributeOfEntity T

… … …

ER schemas

mSMmM

SMM

i-SMi-M

Employees

Departments

EmpNo

Name

Name

Address

P. Atzeni Varna - October 2, 2007 65

Schemas in the supermodel

SM-Abstract

OID Schema Name

301 1 Employees

302 1 Departments

201 3 Clerks

202 3 Offices

SM-AttributeOfAbstract

OID Schema Name isIdent isNullable Type AbstrOID

401 1 EmpNo T F Int 301

402 1 Name F F Text 301

404 1 Name T F Char 302

405 1 Address F F Text 302

501 3 Code T F Int 201

… … … … … … …

MSM-Construct

OID Name IsLex

… … …

3 Abstract F

4 AttributeOfAbstract T

… … …

Supermodel schemas

mSMmM

SMM

i-SMi-M

Employees

Departments

EmpNo

Name

Name

Address

P. Atzeni Varna - October 2, 2007 66

i-SM-Abstract

OID AbsOID InstOID

3010 301 1

3011 301 1

… … …

3010 301 2

i-SM-AttributeOfAbstract

OID AttrOfAbsOID InstOID Value i- AbsOID

4010 401 1 75432 3010

4020 402 1 John Doe 3010

4021 402 1 Bob White 3011

… … … … …

Instances in the supermodel

SM-Abstract

OID Schema Name

301 1 Employees

302 1 Departments

201 3 Clerks

202 3 Offices

SM-AttributeOfAbstract

OID Schema Name isIdent isNullable Type AbstrOID

401 1 EmpNo T F Int 301

402 1 Name F F Text 301

404 1 Name T F Char 302

405 1 Address F F Text 302

501 3 Code T F Int 201

… … … … … … …Supermodel schemasSupermodel instances

mSMmM

SMM

i-SMi-M

P. Atzeni Varna - October 2, 2007 67

Multi-Level Repository, generation and use

model

schema

modelgeneric

modelspecific

des

crip

tion

modelindependence

Supermodel description

(mSM)

Model descriptions

(mM)

Supermodel schemas

(SM)

Model specific schemas

(M)

Supermodel instances

(i-SM)

Model specific instances

(i-M)data

Structure fixed, content provided by tool

designers

Structure generated by the tool from the content of mSM

Structure generated by the tool from the content of mSM

Structure fixed, content provided by model

designers out of mSM

Structure generated by the tool from the content of mM

P. Atzeni Varna - October 2, 2007 68

Translations

• Basic translations are written in a variant of Datalog, with OID invention– We specify them at the schema level– The tool "translates them down" to the data level– Some completion or tuning may be needed

P. Atzeni Varna - October 2, 2007 69

A basic translation

• From (a simple) binary ER model to the relational model– a table for each entity– a column (in the table for E) for each attribute of an entity E– for each M:N relationship

• a table for the relationship• columns …

– for each 1:N and 1:1 relationship:• a column for each attribute of the identifier …

P. Atzeni Varna - October 2, 2007 70

A basic translation application

Departments

Name Address

EmployeesEmpNo

Name

Affiliation

Departments

1,1

0,N

Name

Address

Employees

EmpNo Name Affiliation

P. Atzeni Varna - October 2, 2007 71

A basic translation (in supermodel terms)

• From (a simple) binary ER model to the relational model– an aggregation of lexicals for each abstract– a component of the aggregation for each attribute of abstract– for each M:N aggregation of abstracts …

• …

• From (a simple) binary ER model to the relational model– a table for each entity– a column (in the table for E) for each attribute of an entity E– for each M:N relationship

• a table for the relationship• columns …

– for each 1:N and 1:1 relationship:• a column for each attribute of the identifier …

P. Atzeni Varna - October 2, 2007 72

"An aggregation of lexicals for each abstract"

SM_AggregationOfLexicals(

OID: #aggregationOID_1(OID),

Name: name)

SM_Abstract (

OID: OID,

Name: name ) ;

P. Atzeni Varna - October 2, 2007 73

Datalog with OID invention

• Datalog (informally):– a logic programming language with no function symbols and

predicates that correspond to relations in a database– we use a non-positional notation

• Datalog with OID invention:– an extension of Datalog that uses Skolem functions to

generate new identifiers when needed• Skolem functions:

– injective functions that generate "new" values (value that do not appear anywhere else); so different Skolem functions have disjoint ranges

P. Atzeni Varna - October 2, 2007 74

"An aggregation of lexicals for each abstract"

SM_AggregationOfLexicals(

OID: #aggregationOID_1(OID),

Name: n)

SM_Abstract (

OID: OID,

Name: n ) ;

• the value for the attribute Name is copied (by using variable n)

• the value for OID is "invented": a new value for the function #aggregationOID_1(OID) for each different value of OID, so a different value for each value of SM_Abstract.OID

P. Atzeni Varna - October 2, 2007 75

"An aggregation of lexicals for each abstract"

SM-Abstract

OID Schema Name

301 1 Employees

302 1 Departments

… … …

SM-AttributeOfAbstract

OID Schema Name isIdent isNullable Type AbstrOID

401 1 EmpNo T F Int 301

402 1 Name F F Text 301

… … … … … … …

EmployeesEmpNoName

11

11

Departments1002

Employees1001

SM_AggregationOfLexicals(

OID: #aggregationOID_1(OID),

Name: n)

SM_Abstract (

OID: OID,

Name: n ) ;

3021002

……

1001

SM-AggregationOfLexicals

Schema NameOID

SM-aggregationOID_1_SK

absOIDOID

301

Employees

P. Atzeni Varna - October 2, 2007 76

"A component of the aggregation for each attribute of abstract"

SM_ComponentOfAggregation… ( OID: #componentOID_1(attOID),Name: name, AggrOID: #aggregationOID_1(absOID),IsNullable: isNullable, IsKey: isIdent, Type : type )

←SM_AttributeOfAbstract(

OID: attOID,Name: name,AbstractOID: absOID, IsIdent: isIdent, IsNullable: isNullable , Type : type ) ;

• Skolem functions

– are functions

– are injective

– have disjoint ranges

• the first function "generates" a new value

• the second "reuses" the value generated by the first rule

P. Atzeni Varna - October 2, 2007 77

A component of the aggregation for each attribute of abstract"

SM-Abstract

OID Schema Name

301 1 Employees

302 1 Departments

… … …

EmployeesEmpNoName

11

11

Departments1002

Employees1001

3021002

……

1001

SM-AggregationOfLexicals

Schema NameOID

SM-aggregationOID_1_SK

absOIDOID

301

SM_ComponentOfAggregation… ( OID: #componentOID_1(attOID),Name: name, AggrOID: #aggregationOID_1(absOID),IsNullable: isNullable, IsKey: isIdent, Type : type )

←SM_AttributeOfAbstract(

OID: attOID,Name: name,AbstractOID: absOID, IsIdent: isIdent, IsNullable: isNullable , Type : type ) ;

SM-componentOID_1_SK

absOIDOID

Employees

EmpNo Name

SM-AttributeOfAbstract

OID Schema Name isIdent isNullable Type AbstrOID

401 1 EmpNo T F Int 301

402 1 Name F F Text 301

… … … … … … …

SM-ComponentOfAggregationOfLexicals

Schema Type AggrOIDisNullableisIdentNameOID

Text 1001FFName111004

1001IntFTEmpNo11

1003 401

1004 402

1003

P. Atzeni Varna - October 2, 2007 78

Generating data-level translations

Schema translation

Data translation

• Same environment

• Same language

• High level translation specificationSupermodel description

(mSM)

Supermodel schemas

(SM)

Supermodel instances

(i-SM)

P. Atzeni Varna - October 2, 2007 79

Translation rules, data level

i-SM_ ComponentOfAggregation… (OID: #i-componentOID_1 (i-attOID),i-AggrOID: #i-aggregationOID_1(i-absOID),ComponentOfAggregationOfLexicalsOID:

#componentOID_1(attOID),Value: Value )

←i-SM_AttributeOfAbstract(

OID: i-attOID,i-AbstractOID: i-absOID,AttributeOfAbstractOID: attOID,Value: Value ),

SM_AttributeOfAbstract(OID: attOID,AbstractOID: absOID,Name: attName,IsNullable: isNull,IsID: isIdent,Type: type )

SM_ComponentOfAggregation… ( OID: #componentOID_1(attOID), Name: name, AggrOID:

#aggregationOID_1(absOID),IsNullable: isNullable, IsKey: isIdent, Type : type )

←SM_AttributeOfAbstract(

OID: attOID,Name: name,AbstractOID: absOID, IsIdent: isIdent, IsNullable: isNullable , Type: type ) ;

P. Atzeni Varna - October 2, 2007 80

Correctness

• Usually modelled in terms of information capacity equivalence/dominance (Hull 1986, Miller 1993, 1994)

• Mainly negative results in practical settings that are non-trivial• Probably hopeless to have correctness in general• We follow an "axiomatic" approach:

– We have to verify the correctness of the basic translations, and then infer that of complex ones

P. Atzeni Varna - October 2, 2007 81

Experiments

• A significant set of models– ER (in many variants and extensions)– Relational– OR– XSD– UML

• Demo (good luck!)

P. Atzeni Varna - October 2, 2007 82

XSD

OR noTab ref, FK, nested

Relational

OO ref, nested

OR Tab, ref, FK, nested

OO ref, flat

OR noTab, FK, nested

OR noTab, FK, flat

OR Tab, genref, FK, nested

OR noTab, gen

ref, FK, nested

OR noTab, gen, FK, nested

37+ Remove generalizations36 Unnest sets (with ref)03 Unnest sets (with FKs)24 Unnest structures (flatten)06 Unnest structures (TT & FKs)01 Unnest structures (TT & ref)43 FKs for references29 Tables for typed tables30 Typed tables for tables13 References for FK31 Nest referenced classes

P. Atzeni Varna - October 2, 2007 83

XSD

OR noTab ref, FK, nested

Relational

OO ref, nested

OR Tab, ref, FK, nested

OO ref, flat

OR noTab, FK, nested

OR noTab, FK, flat

OR Tab, genref, FK, nested

OR noTab, gen

ref, FK, nested

OR noTab, gen, FK, nested

36 Unnest sets (with ref)

24 Unnest structures (flatten)

01 Unnest structures (TT & ref)43 FKs for references29 Tables for typed tables

P. Atzeni Varna - October 2, 2007 84

Summary

• ModelGen was studied a few years ago• New interest on it within the "Model management" framework• New approach

– Translation of schema and data– Visible (and in part self generated) dictionary– Visible and modifiable rules– Skolem functions describe mappings

P. Atzeni Varna - October 2, 2007 85

Conclusion

• The ten-year goal of the Asilomar report:– The information utility:

make it easy for everyone to store, organize, access, and analyze the majority of human information online

• A lot of interesting work has been done but …• …integration, translation, exchange are still difficult…• … 2009 is approaching … we are late!

Recommended