Directembedding Concealing the Deep Embedding of DSLs · The good parts 1 Uses standard Scala 2...

DirectembeddingConcealing the Deep Embedding of DSLs

Ólafur Páll Geirsson

École Polytechnique Fédérale de LausanneSchool of Computer and Communication Sciences

June 11, 2015

Ólafur Páll Geirsson (EPFL) Directembedding June 11, 2015 1 / 36

Motivation

Overview

1 Motivation

2 Directembedding

3 Case study: slick-direct v2

4 Conclusion

Motivation

Mission statementEnable wider adoption of embedded DSLs

Motivation

The StruggleDeeply embedded vs. Shallowly embedded

Author User

Shallow

Directembedding

Overview

1 Motivation

2 DirectembeddingArchitectureLanguage virtualizationType overridingImproved error messagesConfiguration

4 Conclusion

Directembedding Architecture

Pipeline

PreProcessing DSL Virtualization

Reification Transformation

Shallow DSL

DeepDSL

PostProcessing

Directembedding Language virtualization

Language Virtualization

Override standard language features

if (cond) 1 else 2 => __ifThenElse(cond, 1, 2)x == y => infix_==(x, y)var x = "str" => __newVar("str")while (cond) body => __while(cond, body)// etc.

Directembedding Type overriding

Type overriding

Override behavior predefined and third-party types

class MyInt {@reifyAs(IntPlus)def +(x: Int): Int = ???

}// Shallowdsl {val x = 1x + 2

}// DeepIntPlus(x, lift(1))

Directembedding Improved error messages

Improved error messages

// Shallowdsl {val s = "foobar"s.charAt(1)

}// Compiler error[error] method charAt on class String is not

supported in example.dsl:s.charAt(1)^

Directembedding Configuration

Configuration@reifyAs

// Inside Query trait.@reifyAs(Take)def take(i: Long): Query[T, C] = ???// Shallow query.query {Query.take(1)

}// Deep query.Take(Query, lift(1))

Configuration@reifyAsInvoked

// Inside Query trait.@reifyAsInvokeddef take(i: Long): Query[T, C] = ???// Shallow query.query {Query.take(1)

}// Deep query.lift(Query).take(lift(1))

Configuration@passThrough

// Inside Query trait.@passThroughdef take(i: Long): Query[T, C]def missingAnnotation(): Int = 1// Shallow query.query {Query.take(missingAnnotation())

}// Deep query.lift(Query).take(missingAnnotation())

Case study: slick-direct v2

Overview

1 Motivation

2 Directembedding

3 Case study: slick-direct v2Lifted embeddingDirect embedding v1Shadow embeddingDirect embedding v2

4 Conclusion

Case study: slick-direct v2

Problem statement

Problem statementDevelop an awesome embedded query language in Scala

Case study: slick-direct v2 Lifted embedding

Lifted embeddingaka Slick

Currently supported API in Slick 3.0

The good parts1 Uses standard Scala2 Feature rich

The bad parts1 Lifted embedding table requires boilerplate2 Cryptic error messages

Problem 1Lifted embedding table requires boilerplate

case class User(id: Int, name: String)

// Lifted embedding tableclass Users(tag: Tag)extends Table[User](tag, "User") {

def id: Rep[Int] = column[Int]("id")def name: Rep[String] = column[String]("name")

def * = ProvenShape.proveShapeOf((id, name) <>((User.apply _).tupled, User.unapply))

Problem 2Cryptic error messages

[Error] User.scala:25: No matching Shape found.Slick does not know how to map the given types.Possible causes: T in Table[T] does notmatch your * projection. Or you use anunsupported type in a Query (e.g. scala List).Required level: slick.lifted.FlatShapeLevel

Source type:(slick.lifted.Rep[Int], slick.lifted.Rep[String],

slick.lifted.Rep[Int])Unpacked type: (Int, String)

Packed type: Anydef * = ProvenShape.proveShapeOf((id, name, id) <> ((User.apply _).tupled,User.unapply))

Case study: slick-direct v2 Direct embedding v1

Direct embeddingDeprecated in Slick 3.0

The good parts1 No more boilerplate2 Comprehensible error messages

The bad parts1 Queries can fail at runtime2 Difficult to develop and maintain

Benefit 1No more boilerplate

@Table("USER")case class User(@PrimaryKey @Column("ID")id: Int,@Column("NAME")name: String

Problem 1Queries can fail at runtime

// Compiles!Query[User].map(_.id.toDouble)// But runtime error...

Case study: slick-direct v2 Shadow embedding

Shadow embeddingPowered by Yin-Yang

Master project of Amir Shaikhha, August 2013

The good parts1 User-friendly API2 Comprehensive and comprehensible error messages3 Great performance

The not so good parts1 Challenging for the DSL author

Shadow embeddingPowered by Yin-Yang

Master project of Amir Shaikhha, August 2013

The good parts1 User-friendly API2 Comprehensive and comprehensible error messages3 Great performance

The not so good parts1 Challenging for the DSL author

We are almost there!

ProblemAuto generated SlickCake

trait YYSlickCakeTuples {type Tuple2[T1,T2] = YYTuple2[T1,T2]implicit def yyRepTuple2ToYYTuple[T1,T2](x:

CakeRep[scala.Tuple2[T1,T2]]): Tuple2[T1,T2]= x.asInstanceOf[Tuple2[T1,T2]]

// 1300 more LOC for up to Tuple22...}trait YYSlickCake with YYSlickCakeTuples {

implicit def yyColumnOptionToYYOption[T](x:YYColumn[scala.Option[T]]): YYOption[T]

= YYOption.fromYYColumn(x)// ...

Problem3 layers of Query

trait Query[T] // Shallowdef map[S](projection: T => S): Query[S]

trait YYQuery[T] // Shadowdef map[S](projection: YYRep[T] => YYRep[S]):

YYQuery[S]= YYQuery.fromQuery(query.map(underlyingProjection(projection))(YYShape.ident[S]))

trait Query[E, T] // Lifteddef map[F, G, T](f: E => F)(implicit shape:

Shape[F, T, G]): Query[G, T]

DirectembeddingFrom shallow to deep in one step

slick-direct v2

Pipeline reminder

Shallow DSL

DeepDSL

PostProcessing

Shallow configurationSimplified example

trait Query[T, C[_]]@reifyAsInvokeddef filter(f: T => Boolean): Query[T, C]

@reifyAs(SlickReification.column _)def column[T, C](e: T, name: String,

tt: TypedType[C]): C@reifyAs(SlickReification.slick_=== _)def infix_==(a: Int, b: Int): Boolean = ???

object SlickPredef@passThroughdef implicitly[T]: T

Deep configurationSimplified example

object SlickReification {def slick_===[T](lhs: lifted.Rep[T], rhs:

lifted.Rep[T]): Rep[Option[Boolean]] = {columnExtensionMethods(lhs) === rhs

}def column[T, C](e: AnyRef,

field: Rep[String],tt: TypedType[C]): Rep[C]

// 20 more LOC}

Directembedding transformationPipeline

Shallow query

query {Query[User].filter(_.name == "Olafur")

Shallow DSL

DeepDSL

PreProcessing

class UserTable extends Table[User] { /* ... */ }UserTable.filter { u =>column(u, "name", implicitly[TypedType[String]])

== "Olafur"}

Shallow DSL

DeepDSL

DSL Virtualization

import DslConfig._class UserTable extends Table[User] { /* ... */ }UserTable.filter { u => infix_==(

shallowColumn(u, "name",implicitly[TypedType[String]]),

"Olafur")

Shallow DSL

DeepDSL

import DslConfig._class UserTable extends Table[User] { /* ... */ }lift(UserTable).filter { u => slick_===(

deepColumn(u, lift("name"),implicitly[TypedType[String]]),

lift("Olafur"))

Shallow DSL

DeepDSL

It works!

FilterSpec

"filter" should "work with string equality" in {equalQueries(query {directUsers.filter(_.name == "Olafur")

}.result,liftedUsers.filter(_.name === "Olafur").result

)} // Success

Conclusion

Overview

1 Motivation

2 Directembedding

4 Conclusion

Conclusion

DirectembeddingLanguage virtualizationType overridingMore reification optionsImproved configuration

Slick-direct v2Under 300 LOC, developed in less than 2 weeksQueries supported: select *, join, filter, map, flatMap, take,(custom column types)

Conclusion

Future work

More case studiesImprove slick-direct

Conclusion

The End

Thank you!

Directembedding Concealing the Deep Embedding of DSLs · The good parts 1 Uses standard Scala 2...

Documents

Tesseract - Concealing Fate (complete guitar and vocal score)

Lev el 3 assets: Booking profits and concealing losses

Agilent De-embedding and Embedding S-Parameter …anlage.umd.edu/Microwave Measurements for Personal Web Site/5980...De-embedding and Embedding S-Parameter Networks Using a ... ial

Embedding Content

The Trouble with Grading: From Concealing to Revealing

Serial RRAM Cell for Secure Bit Concealing

Embedding Design

Website Survival: Concealing Back-End Outages with Oracle Coherence and HotCache

Dowland’s Seven Tears, or the Art of Concealing the · PDF fileDowland’s Seven Tears, or the Art of Concealing the Art By Peter ... “Dowland’s Tears: ... published in 1600

Generating steganographic images via adversarial …...Steganography is a collection of techniques for concealing the existence of information by embedding it within a non-secret medium,

Revealing and concealing: Islamist discourse on human rightsinsct.syr.edu/wp-content/...Concealing.Islamist-Discourse-on-HR3.pdf · Revealing and Concealing: Islamist Discourse on

Automatic pneumatic concealing traffic bollard FAAC … City Pneumatic 220 H600/Faac... · Automatic pneumatic concealing traffic bollard FAAC CITY ... • is built to be integrated

OH 6HFUHWV - Amazon Simple Storage Service · With so many different types of clothes out there, it's ... flatter your figure by showing the good parts and concealing the not so hot

Level 3 assets: Booking proﬁts, concealing losses · Level 3 assets: Booking proﬁts, concealing losses∗ Konstantin Milbradt† July 17, 2010 Abstract Fair value accounting forces

Embedding ea

Negationa Ism in India Concealing the Record of Islam by Koenraad Elst

Pre Embedding

Concealing Cyber-Decoys using Two-Sided Feature Deception ...fferdinando3/cfp/OPTMAS19/papers/… · Concealing Cyber-Decoys using Two-Sided Feature Deception Games 3 Over time, honeypots

Embedding eBook

IS 11992 (1986): Correction fluid for concealing typed errors · IS:11992- 1986 Indian Standard SPECIFICATION FOR CORRECTION FLUID FOR CONCEALING TYPED ERRORS 0. FOR.EWORD 0.1 This