What Can Programming Languages SayAbout Data Exchange?

Michael JohnsonMacquarie University

Michael.Johnson@mq.edu.au

Jorge PérezDCC, Universidad de Chile

jperez@dcc.uchile.cl

James F. TerwilligerMicrosoft Research

james.terwilliger@microsoft.com

ABSTRACTData Exchange, defined generally, is the process of takingdata structured under one schema and transforming it intodata structured under another independent schema. Thisprocess is present in enough scenarios both theoretical andpractical that it has been addressed in many different ways.Most prominent amongst the solutions to the problem is thatproposed by database literature, in which one constructsschema mappings, using (a subset of) first-order predicatecalculus, to establish the high-level relationship among thedatabase schemas participating in the exchange. From aschema mapping an executable process is derived to per-form the exchange. This line of research has made signifi-cant progress and come to impressive findings, but has sometheoretical and practical shortcomings as well. For instance,there are theoretical limitations as to how to compose or in-vert such mappings in a complete and unique way, which is abarrier to making such mappings bidirectional. It is possibleto address some of these shortcomings by looking to solu-tions from a different discipline—a construct from the pro-gramming language literature called a lens—that addressessimilar problems from a different perspective. By combiningsolutions from these two disciplines, one ends up with a newdirection of research as well as a result that might be greaterthan the sum of its parts.

KeywordsBidirectional Transformations, Data Exchange, Lenses

1. INTRODUCTIONData Exchange—the process of taking data in one struc-

ture and processing it into another—is a problem that iswidely seen across nearly all fields of computer science. Fromdatabases (moving data instances between schemas) to visu-alization (moving graphics between formats) to software en-gineering (moving software artifacts between models), manydisciplines face a familiar problem, where given source andtarget descriptions of data:

(c) 2014, Copyright is with the authors. Published in Proc. 17th Inter-national Conference on Extending Database Technology (EDBT), March24-28, 2014, Athens, Greece: ISBN 978-3-89318065-3, on OpenProceed-ings.org. Distribution of this paper is permitted under the terms of the Cre-ative Commons license CC-by-nc-nd 4.0

• How does one specify a transformation of source in-stances into target instances,

• execute that transformation, and

• demonstrate that the transformation has been done asfaithfully as possible?

Even simple data exchange problems can prove to be dif-ficult. For instance, consider a trivial example of mappingdata from a schema Person1(Id,Name,Age,City) to anotherschema Person2(Id,Name,Salary,ZipCode). Some aspectsof this transformation are clear, such as that every instanceof Person1 should correspond to an instance of Person2 withthe same name and identifier post-transformation. However:

• How does one populate the Salary field? Should it befilled in by nulls, or as a function of the ZipCode field?

• How does one populate the ZipCode field? Should it befilled in by nulls, or as a function of the City attributeof the corresponding Person1 instance?

• In the event that one wants to make changes to theinstances in their Person2 form, how are those changesmigrated back to the corresponding Person1 instance?Is the Age field preserved? How does one calculate thevalue of the City attribute?

One could always leave blank the attributes that do notexactly match. In practice, one wants to preserve as muchdata through a transformation as possible. With networkedand cloud-enabled applications, one wants such transforma-tions to be bidirectional to enable updates to propagate be-tween instances. And data-exchange scenarios are rarely sosimple as the example above, as anyone who has written afinancial or healthcare application may attest.

The database field has been developing a solution to thedata exchange problem over the past decade. This solutionuses at its core a subset of first-order predicate calculus,with success in both practical [9] and theoretical applica-tions [1, 11]. Essentially, the transformation is defined byusing a logical implication α //β, in which α and β are first-order predicate calculus expressions, stating that whenevera source database satisfies α then the transformed databaseshould satisfy β. One can use a graphical tool based ondrawing associations using box-and-line diagrams; those di-agrams map cleanly to these predicate calculus expressions.The elegance of this solution and its extensively proven for-mal properties have led to this solution becoming the linguafranca of data exchange for database researchers.

The predicate calculus approach has some notable limi-tations, however. It is commonplace in database research

223 10.5441/002/edbt.2014.21

to take a language or tool designed for one-way mappings(as predicate calculus is) and attempt to reuse it in bidi-rectional scenarios for which is may not be suited. Thereare theoretical results demonstrating that a calculus-baseddata exchange mapping may not have an inverse, and whenit does, that inverse may not be unique [1, 2, 10]. Evenin traditional unidirectional settings, a calculus expressionmay involve existential quantifiers, which mean that targetinstances will need to fill in attributes with labeled nulls.There may be environment information, domain policy, orother sources that one can use to fill in values that are in-accessible to the current formal treatment.

To address these issues, this paper draws from a sequenceof recent efforts to bring together researchers from differentfields, all working on bidirectional transformations [6, 18].Part of the charter of that ongoing effort is to find common-alities between different solutions to similar problems. Onecommonality that has been identified, but only briefly [23], isbetween data exchange and a construct from the program-ming languages community called a lens [14]. This papertakes that germ of an idea and expounds upon it to lay outa potential research path that brings those fields together.

Section 2 briefly describes the state of the art in the dataexchange literature, giving an overview of the formalismused, the successes of the approach, and its limitations. Sec-tion 3 discusses the lens construct in detail, with a partic-ular focus on lens properties that could potentially serve toaddress the limitations from Section 2. In Section 4, the pa-per describes what data exchange mappings could look likewhen leveraging the work of both communities, as well as afew other recent advances. Finally, Section 5 concludes thepaper with some additional thoughts on related work andfuture directions.

2. DATA EXCHANGEIn the database context, data exchange is the problem

that arises when data must be transferred between inde-pendent database applications that do not have the samedata format. In the typical data exchange settings, one isgiven a source schema and a target schema, usually rela-tional schemas, a schema mapping M that specifies the rela-tionship between the source and the target, and a databaseinstance I of the source schema. The basic problem thatone wants to address is how to materialize an instance ofthe target schema that reflects the source data as accuratelyas possible [1, 11].

The building block of data exchange, as well as of manyother data-interoperability tasks, is the notion of schemamapping, which is a high-level specification, usually statedin a logical language, that describes the relationship betweendatabase schemas. The most extensively studied schema-mapping language in the database literature, is the languageof source-to-target tuple-generating dependencies (st-tgds).An st-tgd is a sentence of the form

∀x(∃yϕS(x, y) // ∃zψT (x, z)

), (1)

where ϕS(x, y) and ψT (x, z) are conjunctions of relationalatoms (names of tables with variables) over the source andtarget schemas, respectively. The semantics of st-tgds isinherited from the semantics of first order logic. A pair ofdatabase instances I and J of the source and target schemas,respectively, satisfies (1) whenever the following holds for ev-ery tuple a: if formula ϕS(a, b) is true in I for some elements

Figure 1: Visual representation of st-tgds

b, then there exists elements c such that ψT (a, c) is true inJ . Moreover, given a fixed I and a set of st-tgds M, everytarget instance J such that (I, J) satisfies all the st-tgds inM, is called a solution for I under M. Thus, given I andM, the data exchange problem can be formalized as how tomaterialize the best solution for I under M.

Example 1. Consider a source schema composed of asingle relation Emp(·) to store employees, and a target schemaManager(·, ·). A possible st-tgd between these schemas is

∀x(Emp(x) // ∃y Manager(x, y)

), (2)

which essentially states that every employee in the sourceschema should have a manager in the target schema.

Consider a source instance I = {Emp(Alice), Emp(Bob)}.Two possible solutions for I under the mapping specified bythe above st-tgd are

J1 = {Manager(Alice,Alice), Manager(Bob,Alice)},J2 = {Manager(Alice,Bob), Manager(Bob,Ted)}.

Notice that there is no constraint over the possible valuesused as managers in the target. Actually the following isalso a solution

J∗ = {Manager(Alice,⊥1), Manager(Bob,⊥2)},where ⊥1 and ⊥2 are labelled nulls used to represent thata value should be there but the mapping does not provideenough information to completely determine it. Actually,J∗ is considered as the preferred solution for the exchangeas it is the most general among all the possible solutions [1].

In a practical setting, an end user does not directly specifya mapping by writing down an st-tgd, but by specifyingsome simple correspondences usually exploiting some visualinterface [9]. Figure 1 shows an example of correspondencesestablished by drawing arrows between database schemas.These visual representations are then compiled into sets ofst-tgds. For example, the upper part in Figure 1 can berepresented by the st-tgd

∀x∀y(Takes(x, y) // ∃z(Student(z, x) ∧ Assgn(x, y))

),

while the lower part is represented by

∀x∀z(∃y(Student(x, y)∧Assgn(y, z)) //Enrollment(x, z)

).

In the last few years, a lot of attention has been paid tothe development of solid foundations for the problem of ex-changing data using schema mappings. These developments

224

(M!)"1 ! M

A B

A!

M

M!

Figure 1: The schema evolution problem

obtained by inverting mapping M! and then composing theresult with mapping M.

The intuitive description of composition and inversion pre-sented in the previous paragraph has turned out to be verydi!cult to formalize, and several di"erent semantics havebeen proposed in the last few years [18, 16, 19, 2, 4]. For ev-ery semantics, several questions need to be answered, amongthem, questions about computability, existence, and expres-sivility. This last topic is considerably important in practice.Given that mappings are usually specified using st-tgds, onemain question is whether the composition of two of thesemappings can also specified by using st-tgds. A similar ques-tion arises for the inverse operator. It has been shown thatst-tgds are not strong enough to specify neither compositionnor inversion of st-tgds for most of the semantics proposedso far in the literature. This has motivated the search fora closed mapping language, a language L such that the re-peated application of the composition and inversion opera-tors over mappings specified in L, can always be specified inL. The existence of such a language is still one of the mostchallenging open problems in the area.

Several features of st-tgds are in the focus of the prob-lems mentioned above. For instance, the existential quan-tification both in the source and the target-side of mappings,make them non-deterministic which di!cults not only theprocesses of exchanging data, but also composing and in-verting mappings. Actually, Fagin et al. showed that thelanguage of st-tgds that do not use existential quantifica-tion in the target side, is closed under composition [18].Similarly, Arenas et al. proposed a weak semantics for in-version for which the extension of st-tgds with inequalities(plus a predicate to di"erentiate nulls from constant values)is closed under inversion [4]. Nevertheless, the search for alanguage closed under both operators has been somewhatelusive for the database community.

Although the data exchange problem has been success-fully studied from a database perspective, several challengesremain open, among the most important, to find a practi-cal mapping language with good properties for specifyingexchange processes, and at the same time with good prop-erties for computing and expressing high-level schema map-ping operators. We think that in the search for answers tothese open questions, the database community should opento new ways of attacking the problem, borrowing tools andtechniques from areas that have faced similar challenges butfrom di"erent perspectives.

4. LENSESWhat is a lens? What are its properties? Symmetric

versus asymmetric?

What are relational lenses? (AKA, how do lenses workwith relational data?)

How are lenses perfectly suited to the data exchange prob-lem?

What bits do we need to add to lenses to meet the“perfectrequirements” in the previous section?

5. HOW THE WORLD COULD BE, AND WHATWE NEED TO DO IT

Use a tool to specify the mapping. Tool, like Clio, has anatural translation of state to st-tgds.

The inspiration from a new approach comes from an oldidea in the relational database world, namely the relation-ship between relational calculus and relational algebra fa-miliar to every database researcher and central to every re-lational database implementation. The SQL language waslargely inspired by domain relational calculus and was in-tended as a declarative specification to be later translatedinto operational atoms. Specifically, the user workflow is asfollows:

• (optional) Via some form of query builder, the userconstructs a query that is translated into SQL.

• The SQL query is translated statically to a relationalalgebra expression.

• The relational algebra expression is translated to aquery plan by associating algorithms with operators,and by applying optimization routines. This processis highly informed by gathered statistics, and may beinformed by user intervention.

By associating st-tgds with relational lenses, one can imag-ine a similar workflow in a bidirectional or synchronizationworld:

• Via some form of visual interface, the user constructsa mapping that is translated into st-tgds.

• The collection of st-tgds is translated statically to arelational lens template.

• The relational lens template is translated to a mappingplan by associating algorithms with operators, and byapplying optimization routines. This process is highlyinformed by gathered statistics, and in some instancesmust be informed by user intervention.

Supplement the specification with some additional hints,like “what do I do with this extra column” or “if I changethis bit over here, how should I change the mapping or othermodel”.

Need a st-tgd-to-lens compiler. Need a completeness proof.

6. CONCLUSIONLots of work to do, which is good. Work is interdisci-

plinary. Work is preliminary but visionary. If you don’taccept this paper, then you don’t like freedom.

7. REFERENCES[1] M. Arenas, P. Barcelo, L. Libkin, and F. Murlak.

Relational and XML Data Exchange. Morgan andClaypool Publishers, 2010.

Figure 2: The schema evolution problem

are a first step towards providing a general framework forexchanging information, but they are definitely not the lastone. In fact, many information system problems involve notonly the design and integration of complex application ar-tifacts, but also their subsequent manipulation. This hasmotivated the need for the development of a general infras-tructure for managing schema mappings. In particular, sev-eral operators for manipulating schema mappings have beenshown to be fundamental in this area.

Two of the most fundamental operators on schema map-pings are composition and inversion [3]. Given a mappingM1 from a schema A to a schema B, and a mapping M2

from B to a schema C, the composition of M1 and M2

is a new mapping that describes the relationship betweenschemas A and C. This new mapping must be semanticallyconsistent with the relationships previously established byM1 andM2. On the other hand, an inverse ofM1 is a newmapping that describes the reverse relationship from B toA, and is semantically consistent withM1. In practical sce-narios, these operators can have several applications one ofthem being the schema evolution problem (Figure 2). Con-sider a mappingM between schemas A and B, and assumethat schema A evolves into a schema A′. This evolutioncan be expressed as a mappingM′ between A and A′. Therelationship between the new schema A′ and schema B canbe obtained by inverting mapping M′ and then composingthe result with mapping M.

The intuitive description of composition and inversion pre-sented in the previous paragraph has turned out to be verydifficult to formalize, and several different semantics havebeen proposed in the last few years [2, 4, 10, 12, 13]. For ev-ery semantics, several questions need to be answered, amongthem, questions about computability, existence, and expres-siveness. This last topic is very important in practice. Giventhat mappings are usually specified using st-tgds, one mainquestion is whether the composition of two of these map-pings can also be specified by using st-tgds. A similar ques-tion arises for the inverse operator. As shown in the follow-ing example st-tgds are not strong enough to specify neithercomposition nor inversion of st-tgds for most of the seman-tics proposed so far in the literature.

Example 2 ([12]). LetM be the mapping specified bythe st-tgd (2) in Example 1. Consider a new schema com-posed of relations Boss(·, ·) and SelfMngr(·) and the st-tgds

∀x∀y(Manager(x, y) // Boss(x, y)

),

∀x(Manager(x, x) // SelfMngr(x)

).

It can be shown [12] that if the mapping in Example 1 iscomposed with the above mapping, the result can be speci-

fied by the logical sentence

∃f[

∀x(Emp(x) // Boss(x, f(x))

)∧

∀x(Emp(x) ∧ x = f(x) // SelfMngr(x)

) ].

This sentence essentially states that there exists a functionf(·) that assigns a manager/boss to every employee, andmoreover, if the manager/boss assigned to an employee eequals f(e), then e should be in the table SelfMngr. Thislogical formula is not an st-tgd, actually it is not even infirst-order logic, as it has a second-order quantification overa function symbol. It was shown by Fagin et al. [12] thatsecond-order quantification, plus the equality in the left sideof the above formula, is unavoidable if one wants to faithfullyrepresent the composition of the two mappings. That is, acomposition specified by st-tgds does not exist in this case.

Example 3. To show some of the problems that arisewith inversion, consider a mapping specified by the st-tgds

∀x∀y(Father(x, y) // Parent(x, y)

),

∀x∀y(Mother(x, y) // Parent(x, y)

),

and let I = {Father(Leslie,Alice)}. Then the best solutionfor I is the instance J = {Parent(Leslie,Alice)}. When try-ing to establish the reverse relationship, it is not clear whatsource instance should be assigned to J . In fact, accordingto Fagin’s initial definition of inverse [10], the above map-ping is not invertible. Subsequent works on relaxed notionsof inverses for mappings have tried to solve this problem byapplying a best effort approach [2, 13]. In particular, underthe definition of Arenas et al. [2], the best possible inversefor the above mapping is given by the sentence

∀x∀y(Parent(x, y) // Father(x, y) ∨ Mother(x, y)

).

Notice that under the inverse mapping both instances I1 ={Father(Leslie,Alice)} and I2 = {Mother(Leslie,Alice)}, areequally good as solutions for J = {Parent(Leslie,Alice)}.Also notice that the above mapping is not an st-tgd as ithas a disjunction in the right-hand side, which was shown byArenas et al. [2] to be unavoidable in this case. This exampleshows that inverses in general may lose information whengoing back from target to source, and more importantly,that inverses specified by st-tgds might not exist.

The fact that st-tgds cannot faithfully represent the com-position or inversion of st-tgds has motivated the search fora closed mapping language; a language L such that the re-peated application of the composition and inversion opera-tors over mappings specified in L, can always be specified inL. The existence of such a language is still one of the mostchallenging open problems in the area.

Several features of st-tgds are in the focus of the problemsmentioned above. For instance, the existential quantifica-tion both on the source and the target-side of mappings,makes them non-deterministic which makes more difficultnot only the processes of exchanging data, but also compos-ing and inverting mappings. Actually, Fagin et al. showedthat the language of st-tgds that do not use existential quan-tification on the target side, is closed under composition [12].Similarly, Arenas et al. proposed a weak semantics for in-version for which the extension of st-tgds with inequalities(plus a predicate to differentiate nulls from constant values)is closed under inversion [4]. Nevertheless, the search for a

225

language closed under both operators has been somewhatelusive for the database community. An addition to st-tgdswhich has been very common in the literature are target de-pendencies used to enforce properties only over the targetside (for example, key and foreign-key constraints on thetarget). Target dependencies add expressive power and canbe used to decrease the level of non-determinism when ex-changing data, but at the same time, they complicate themanaging of mappings at a high level.

Although the data exchange problem has been success-fully studied from a database perspective, several challengesremain open, among the most important, to find a practi-cal mapping language with good properties for specifyingexchange processes, and at the same time with good prop-erties for computing and expressing high-level schema map-ping operators. We think that in the search for answersto these open questions, the database community should beopen to new ways of attacking the problem, borrowing toolsand techniques from areas that have faced similar challengesbut from different perspectives.

3. LENSESLenses [14] are a useful technique developed in the pro-

gramming language community (although it is noteworthythat some lens concepts appear in work in other communi-ties including work on data storage and caching). Lenseshave played an important role in an international researchinitiative aimed at developing and understanding techniquesfor bidirectional transformations [6, 18].

The motivation for the development of lenses closely par-allels some of the open issues raised in the previous section.Bidirectional transformations, unless they happen to be bi-jections,

• have to provide a candidate for a “reverse” transfor-mation, even when an inverse transformation doesn’texist

• have to deal with potential non-determinism in the re-verse transformation

• should be closed under composition, and ultimately

• should form a closed mapping language as describedabove.

Lenses, in their various guises, provide such bidirectionaltransformations.

The most basic form of a lens, called a set-based lens, con-sists of two sets S and V and two functions g (pronouncedget) S //V , and p (pronounced put) V ×S //S. Set-basedlenses are asymmetric and can be thought of as an abstrac-tion of a view updating problem — S should be thought ofas the set of states of a system, and V as the set of statesof a view of that system. Since a view state contains lessinformation than a system state, a view state can easily becalculated from any system state, and that’s what g does.However given just a view state it is not in general possibleto determine a single corresponding system state because ofthe missing information. Instead, the lens provides a way, p,of calculating a new system state from an old system stateand a specified view state. The function p can be thoughtof as view updating : Given a system state s and its corre-sponding view g(s), if the view is modified to some new viewstate v, then the system state s should be updated to a newsystem state p(v, s).

A lens is called well-behaved if it satisfies two conditions

(i) (PutGet) the updated system state really does corre-spond to the view state v: g(p(v, s)) = v, and

(ii) (GetPut) the Put for a trivially updated state is trivial:p(g(s), s) = s.

Some examples of work on well-behaved asymmetric lensesinclude quotient lenses [15], which allow the properties of alens to be relative to equivalence classes; delta lenses [8,21], which enrich the situation by using the nature of themodification, the delta, from g(s) to v to compute a deltawhich can be used to update s; edit lenses [16], which take asinput edit operations rather than simple deltas; and so on.In each case the lenses are composable, and lenses are in asense a schema mapping, but, sadly, inversion of asymmetriclenses is, except in trivial cases, not defined.

These lenses are all called asymmetric to emphasise thedifference between S and V : S contains all the informa-tion and V some information that can be derived from S.The lenses themselves are sometimes said to go from S toV . They are bidirectional, but asymmetrically so. Data ex-change, and many other notions of data synchronisation, arein contrast symmetric — there is no master source of datalike S, and no a priori “easy” direction for data exchange.

Recently, a notion of well-behaved symmetric lens hasbeen introduced [17]. Although they are defined elementar-ily, a symmetric lens can be seen to be equivalent to a spanof asymmetric lenses: A set-based symmetric lens betweenS and T amounts to a set U (sometimes called “universal”because it contains all the information of both S and T , andin general even more besides) and two asymmetric lenses,one from U to S and one from U to T . Symmetric lensesare composable and each symmetric lens has an inversionobtained by exchanging the roles of S and T .

(A span is, as just described, a pair of lenses (or functions,or more generally other agreed operations) with a commonstarting position viz S oo U // T . A cospan (cospans willbe referred to at the end of the paper) is similarly a pair oflenses (or functions, etc) with common ending positions vizS //X oo T .)

There is an analogy here with functions and relations —functions do not in general have uniquely defined inversions,but instead we might consider spans of functions that formgeneralised relations and do have inversions given by revers-ing the relation. The analogy is in fact the mathematicalbasis of the symmetric lens constructions: in both cases in-version is obtained the same way, and composition is calcu-lated by pullback (the basis of relational composition).

As a concrete example, consider the work to date on rela-tional lenses [5]. Relational lenses have a strong correlationwith relational algebra; for instance, there is a “projection”lens corresponding to the projection operator π. Becauseeach lens must specify put and get functions p and g, eachlens not only describes how to retrieve data as does its re-lational algebra counterpart, but also how to update andreplace it. Thus, relational lenses can be considered a solu-tion to the view-update problem [7].

Each given relational operator does not have a unique up-date policy, however. For the given simple example of theprojection operator π, if the operator drops a column c, anda new row is added to the output (view) state, there are sev-eral possibilities as to how to populate that column c whenadding the row to the input state. Some possibilities are:

• Always use a null value.

226

• Always use a constant value.

• Always insert an environment value, e.g., the currenttime or user.

• Use a functional dependency c′ → c from another col-umn c′ to determine the value.

The original work on relational lenses treats the last ofthose options as the proper one in the sense that it is theleast lossy, but requires the presence of a functional depen-dency to operate. Each of these choices of update policyis equally valid based on the requirements of the user andthe available data. Therefore, one can equally consider arelational lens template as a way to describe a family of po-tential lenses corresponding to a specific relational operatorbut missing its update policy.

The update policy for the projection operator is very sim-ilar to a way to specify how to resolve existential quantifiersin st-tgds, but update policies for other operator templatesneed not be. For example, the join and union lens templatesmust have update policies specifying whether updates arepropagated to the left or right inputs, or to both.

It is important to note that relational lenses to date areasymmetric. As noted before, to be a complete solution todata exchange scenarios, an important first step would beto develop symmetric versions of these lenses. However, thepotential association between lenses, especially relationallenses, and data exchange is powerful. There is a rich andevolving body of work on lenses and their generalisations.Lenses are an abstraction of schema mappings. And sym-metric lenses provide a closed mapping language since theyhave inversions and compositions. Thus lenses seem set tosolve outstanding problems in data exchange including thedevelopment of a closed mapping language and the schemaevolution problem. Furthermore, of course, the developmentof lenses will likewise be positively influenced by closer andmore explicit links with the database community and prac-tical data exchange.

4. BRINGING THEM TOGETHERWhat is hopefully clear by this point is that the data ex-

change problem is relevant to multiple fields of study withincomputer science, and that different fields have developedsolutions to the problem that could potentially learn fromeach other. To summarize:

An st-tgd is a declarative, unidirectional language. It hastool support in the form of mapping builders [9]. Theformal treatment of st-tgds demonstrates a great manypositive angles, but has practical problems when itcomes to existential quantifiers and completeness un-der inversion and composition.

Lenses combine to become operationally specified, domain-specific bidirectional languages. One such language isrelational lenses, which have general parity with rela-tional algebra.

There are almost certainly many possible ways to inter-act between these two tools and formalisms. However, oneconcrete approach comes from an old idea in the relationaldatabase world, namely the relationship between relationalcalculus and relational algebra familiar to every databaseresearcher and central to every relational database imple-mentation. The SQL language was largely inspired by do-main relational calculus and was intended as a declarative

specification to be later translated into operational atoms.Specifically, the user workflow is as follows:

• (optional) Via some form of query builder, the userconstructs a query that is translated into SQL.

• The SQL query is translated statically to a relationalalgebra expression.

• The relational algebra expression is translated to aquery plan by associating algorithms with operators,and by applying optimization routines. This processis highly informed by gathered statistics, and may beinformed by user intervention.

By associating st-tgds with lenses, one can imagine a sim-ilar workflow in a bidirectional or synchronization world:

• Via some form of visual interface, the user constructsa mapping that is translated into st-tgds.

• The collection of st-tgds is translated statically to arelational lens template.

• The relational lens template is translated to a mappingplan by associating algorithms with operators, and byapplying optimization routines. This process is highlyinformed by gathered statistics, and in some instancesmust be informed by user intervention (in the samespirit as past work on updatable views [22]).

An added benefit to this approach is that a mapping wouldnow have a “show plan” capability similar to that used in re-lational database engines. The designer of a mapping wouldbe able to see not only how the mapping is specified (inlanguage that is natural to st-tgds) but also how it will beevaluated (in language that is natural to anyone that hasever used a relational database).

One necessary difference between this approach and therelationship between SQL and relational algebra is the ex-pected amount of necessary user intervention. With a typi-cal relational engine, user hints are entirely optional; a usermay be able to specify indexes or join algorithms, but with-out any such input, the engine still manages to determinean optimal query plan a vast majority of the time. Withthe data exchange scenario, one would need to somehow fillin the relational lens template parameters, needing answersto questions like “what do I do with this extra column”.While reasonable defaults may exist, it is unclear as to howoften those defaults will be optimal to the user’s scenarios.Therefore, it is important to discover some way to map thosequestions to user input, which is primarily a usability ques-tion, and a non-trivial one at that. Thus, to successfullyincorporate the two technologies along these lines, at thevery least the following will need to be accomplished:

1. A symmetric version of the relational lens framework.

2. An st-tgd-to-lens compiler, and a completeness proofof that compiler.

3. A reasonable mapping of relational lens template pa-rameters to user gestures — for instance, giving theuser an understandable way to dictate through whichinputs an update to a join should propagate.

One final solution of note is that one can use a combi-nation of lenses and st-tgds to solve the schema evolutionproblem in Section 2 as well, and possibly in two different

227

manners. The simplest of the two possible approaches isto note that composing mappings specified using lenses isas simple as concatenating them. So, if there is a mappingfrom S to T as [m1,m2,m3], and one can express a schemaevolution operation against S to S′ as a sequence of sym-metric lenses [`1, `2], then one can construct a mapping fromS′ to T as [`−1

2 , `−11 ,m1,m2,m3].

It should be noted that lenses are not the only recent at-tempt to construct mappings from incremental componentssuch [23]; one such language (called channels) allows schemaevolution primitives to be propagated through mappingsrather than appended to one end [24]. It may prove usefulto end users that have control over both ends of a mappingto have a choice between adapting one schema and compos-ing the mappings as in the previous paragraph, or propagatethe evolution primitives through the mapping and constructa new, evolved target schema T ′. To support the latter,one must add an additional research goal to the above list,namely to update the lens formalism to allow the possibil-ity of schema evolution, or otherwise resolve the differencesbetween lenses and channels.

5. CONCLUSIONThis paper has described the status quo of research on

data exchange on two different fronts in two different dis-ciplines: st-tgds from database literature, and lenses fromprogramming language literature. Both tools have rich pub-lication histories and formal results; the vision laid out inthis paper is not intended to replace either one. Rather, itseems likely that by combining the work done in both areasand following the spelled-out research agenda, a novel, com-plete, and practical solution to the data exchange problemmay result. To realize the vision will require work at practi-cal and formal levels and across interdisciplinary boundaries.

There are also tantalising opportunities that require fur-ther study. Among them there is work showing how certainkinds of lenses really do correspond concretely to schemamappings [20]. Can this be generalised to symmetric lensesin their various forms? Are symmetric lenses really an em-bodiment of schema mappings with a built-in calculus ofdata exchange? Also, there is already practical work inbuilding data exchange via cospans of certain kinds of lenses[19]. That work has been used to concretely implementdata exchange and systems interoperation. A co-span ofasymmetric lenses is not a symmetric lens, but there mustbe a precise mathematical relationship between the co-spanbased data exchange and the span based schema mappings.

6. REFERENCES[1] M. Arenas, P. Barcelo, L. Libkin, F. Murlak. Relational

and XML Data Exchange. Synthesis Lectures on DataManagement, Morgan and Claypool Publishers, 2010.

[2] M. Arenas, J. Perez, C. Riveros. The recovery of aschema mapping: bringing exchanged data back. TODS34(4) (2009).

[3] M. Arenas, J. Perez, J.L. Reutter, C. Riveros.Composition and inversion of schema mappings.SIGMOD Record 38(3), 17–28 (2009).

[4] M. Arenas, J. Perez, J.L. Reutter, C. Riveros. QueryLanguage based Inverses of Schema Mappings:Semantics, Computation, and Closure PropertiesVLDBJ 21(6), 823–842 (2012)

[5] A. Bohannon, B.C. Pierce, J.A. Vaughan. Relationallenses: a language for updatable views. PODS 2006.

[6] K. Czarnecki, J.N. Foster, Z. Hu, R. Lammel, A.Schurr, J.F. Terwilliger. Bidirectional Transformations:A Cross-Discipline Perspective. ICMT 2009.

[7] U. Dayal and P. Bernstein. On the Correct Translationof Update Operations on Relational Views. ACMTransactions on Database Systems, September 1982,8(3).

[8] Z. Diskin, Y. Xiong, K. Czarnecki. From State- toDelta-Based Bidirectional Model Transformations: theAsymmetric Case, Journal of Object Technology 2011,10(6), 1–25.

[9] R. Fagin, L.M. Haas, M.A. Hernandez, R.J. Miller, L.Popa, Y. Velegrakis et al. Clio: Schema MappingCreation and Data Exchange. Conceptual Modeling:Foundations and Applications, 2009, 198–236.

[10] R. Fagin. Inverting schema mappings. TODS 32(4)(2007).

[11] R. Fagin, P.G. Kolaitis, R.J. Miller, L. Popa. Dataexchange: semantics and query answering. TCS 336(1),89–124 (2005).

[12] R. Fagin, P.G. Kolaitis, L. Popa, W.C. Tan.Composing schema mappings: Second-orderdependencies to the rescue. TODS 30(4), 994–1055(2005).

[13] R. Fagin, P.G. Kolaitis, L. Popa, W.C. Tan.Quasi-inverses of schema mappings. TODS 33(2)(2008).

[14] J.N. Foster, M.B. Greenwald, J.T. Moore, B.C. Pierce,A. Schmitt. Combinators for bidirectional treetransformations: A linguistic approach to theview-update problem. ACM Trans. Program. Lang.Syst., 29(3): (2007).

[15] J.N. Foster, A. Pilkiewicz, B.C. Pierce. QuotientLenses. ICFP 2008.

[16] M. Hofmann, B.C. Pierce, D. Wagner. Edit lenses.POPL 2012.

[17] M. Hofmann, B.C. Pierce, D. Wagner. Symmetriclenses. POPL 2011.

[18] Z. Hu, A. Schurr, P. Stevens, J.F. Terwilliger.Dagstuhl seminar on bidirectional transformations(BX). SIGMOD Record 40(1), (2011).

[19] M. Johnson. Enterprise Software with Half-DuplexInteroperations. In Doumeingts, Mueller, Morel andVallespir (eds), Enterprise Interoperability: NewChallenges and Approaches, 521–530, Springer-Verlag,(2007).

[20] M. Johnson and R. Rosebrugh. Fibrations andUniversal View Updatability. Theoretical ComputerScience, 388, 109–129, (2007).

[21] M. Johnson and R Rosebrugh. Delta lenses andopfibrations. BX 2013, to appear.

[22] A.M. Keller. Choosing a View Update Translator byDialog at View Definition Time. VLDB 1986, 467–474.

[23] J.F. Terwilliger, A. Cleve, C. Curino. How Clean IsYour Sandbox? ICMT 2012.

[24] J.F. Terwilliger, L.M.L. Delcambre, D. Maier, J.Steinhauer, S. Britell. Updatable and EvolvableTransforms for Virtual Databases. PVLDB 3(1)(VLDB 2010).

228