Practical Groovy Domain-Specific Languages

Practical Domain-Specific Languages in Groovy

Guillaume LaforgeGroovy Project ManagerSpringSource

2009 CommunityOne Conference: EAST | developers.sun.com/events/communityone

Groovy Project Manager

JSR-241 Spec LeadHead of Groovy Developmentat SpringSource

Initiator of the Grails frameworkCo-author of Groovy in Action

Speaker: JavaOne, QCon, JavaZone, Sun TechDays, Devoxx, The Spring Experience, JAX, Dynamic Language World, IJTC, and more...

Guillaume Laforge

2


A few words about Groovy

Groovy is a dynamic language for the JVM• with a Meta Object Protocol

• compiles directly to bytecode, seamless Java interop

Open Source ASL 2 project hosted at Codehaus

Relaxed grammar derived from Java 5• + borrowed good ideas from Ruby, Python, Smalltalk

Fast... for a dynlang on the JVM

Closures, properties, optional typing, BigDecimal by default, nice wrapper APIs, and more...

3


The context and the usual issues we faceSome real-life examples of Domain-Specific LanguagesGroovy’s DSL capabilitiesIntegrating a DSL in your applicationConsiderations to remember when designing your own DSL

Agenda

4

The context

Subject Matter Experts,Business analysts...

HAICAN HAS STDIO?I HAS A VARIM IN YR LOOP UP VAR!!1 VISIBLE VAR IZ VAR BIGGER THAN 10? KTHXBYEIM OUTTA YR LOOPKTHXBYE

Developer producing LOLCODE

Lots of languages...

And in the end......nobody understands each other

Expressing requirements...


DSL: a potential solution?

Use a more expressive language than a general purpose one

Share a common metaphore of understanding between developers and subject matter experts

Have domain experts help with the design of the business logic of an application

Avoid cluttering business code with too much boilerplate technical code

Cleanly separate business logic from application code

Let business rules have their own lifecycle

11


Towards more readibility (1)

12



12



20%

12



13



13



80%

13



Agenda

14


A collection of DSLs

In our everyday life, we’re surrounded by DSLs

• Technical dialects

• Notations

• Business languages

15


Technical dialects

16

SQL


^[\w-\.]+@([\w-]){2,4}$

18


Notations

19

1. e4 e52. Nf3 Nc63. Bb5 a6

L2 U F-1 B L2 F B -1 U L2

Visual!


Business languages

23


Real-life Groovy examples

Anti-malaria drug resistance simulationHuman Resources employee skills representationInsurance policies risk calculation engineLoan acceptance rules engine for a financial platformMathematica-like lingua for nuclear safety simulationsMarket data feeds evolution scenarios

and more...

24



Agenda

25


A flexible & malleable syntax

No need to write full-blown classes, use scriptsOptional typing (def)• in scripts, you can even omit the def keyword

Native syntax constructsParentheses & semi-colons are optionalNamed argumentsBigDecimal by default for decimal numbersClosures for custom control structuresOperator overloading

26


Scripts vs classes

Hide all the boilerplate technical code• an end-user doesn’t need to know about classes

•public class Rule { public static void main(String[] args) { System.out.println(“Hello”); }}

•println “Hello”

27


Optional typing

No need to bother with types or even generics• unless you want to!

Imagine an interest rate lookup table method returning some generified type:

•Rate<LoanType, Duration, BigDecimal>[] lookupTable() { ... }def table = lookupTable()

No need to repeat the horrible generics type info!

28


Native syntax constructs

Lists• [Monday, Tuesday, Wednesday]

Maps• [CA: ‘California’, TX: ‘Texas’]

Ranges• def bizDays = Monday..Friday• def allowedAge = 18..65• You can create your own custom ranges

29


Optional parens & semis

Make statements and expressions look more like natural languages

• move(left);

• move left

30


Named arguments

In Groovy you can mix named and unnamed arguments for method parameters

• named params are actually put in a map parameter• plus optional parens & semis

take 1.pill, of: Chloroquinine, after: 6.hours

Corresponds to a method signature like:

•def take(Map m, MedicineQuantity mq)

31


BigDecimal by default

Main reason why financial institutions often decide to use Groovy for their business rules!

• Although these days rounding issues are overrated!

Java vs Groovy for a simple interpolation equation

BigDecimal uMinusv = c.subtract(a); BigDecimal vMinusl = b.subtract(c); BigDecimal uMinusl = a.subtract(b); return e.multiply(uMinusv) .add(d.multiply(vMinusl)) .divide(uMinusl, 10, BigDecimal.ROUND_HALF_UP);

(d * (b - c) + e * (c - a)) / (a - b)

32


Custom control structures,thanks to closures

When closures are last, they can be put “out” of the parentheses surrounding parameters

unless (account.balance > 100.euros, { account.debit 100.euros })

unless (account.balance > 100.euros) { account.debit 100.euros}

Signature def unless(boolean b, Closure c)

33


Operator overloading

Currency amounts•15.euros + 10.dollars

Distance handling•10.kilometers - 10.meters

Workflow, concurrency•taskA | taskB & taskC

Credit an account•account << 10.dollarsaccount += 10.dollarsaccount.credit 10.dollars

a + b a.plus(b)

a - b a.minus(b)

a * b a.multiply(b)

a / b a.divide(b)

a % b a.modulo(b)

a ** b a.power(b)

a | b a.or(b)

a & b a.and(b)

a ^ b a.xor(b)

a[b] a.getAt(b)

a << b a.leftShift(b)

a >> b a.rightShift(b)

+a a.positive()

-a a.negative()

~a a.bitwiseNegate()34

Groovy’s dynamic heart:

The MOP!MetaObject Protocol


Groovy’s MOP

All the accesses to methods, properties, constructors, operators, etc. can be intercepted thanks to the MOP

While Java’s behavior is hard-wired at compile-time in the class

Groovy’s runtime behavior is adaptable at runtime through the metaclass.

Different hooks for changing the runtime behavior• GroovyObject, custom MetaClass implementation, categories,

ExpandoMetaClass

36


GroovyObject

All instances of classes created in Groovy implement the GroovyObject interface:

•getProperty(String name)•setProperty(String name, Object value)

•invokeMethod(String name, Object[] params)

•getMetaClass()•setMetaClass(MetaClass mc)

A GO can have “pretended” methods and properties

37


MetaClass

The core of Groovy’s MOP system

•invokeConstructor()•invokeMethod() and invokeStaticMethod()

•invokeMissingMethod()•getProperty() and setProperty()

•getAttribute() and setAttribute()•respondsTo() and hasProperty()

MetaClasses can change the behavior of existing third-party classes — even from the JDK

38


ExpandoMetaClass

A DSL for MetaClasses!

MoneyAmount.metaClass.constructor = { ... }Number.metaClass.getDollars = { ... }Distance.metaClass.toMeters = { ... }Distance.metaClass.static.create = { ... }

To avoid repetition of Type.metaClass, you can pass a closure to metaClass { ... }

The delegate variable in closure represents the current instance, and it the default parameter

39

The Builder pattern


A builder for HR

softskills { ideas { capture 2 formulate 3 } ...}knowhow { languages { java 4 groovy 5 } ...}

41


A builder for HR

softskills { ideas { capture 2 formulate 3 } ...}knowhow { languages { java 4 groovy 5 } ...}

41


Builders

Builders are... • a mechanism for creating any tree-structered graph• the realization of the GoF builder pattern at the syntax level in Groovy

• simply a clever use of chained method invocation, closures, parentheses omission, and use of the GroovyObject methods

Existing builders• XML, Object graph, Swing, Ant, JMX, and more...

42


The clever trick

GroovyObject#invokeMethod() is used to catch all non-existing method calls in the context of the builder

The nesting of closures visually shows the level of nesting / depth in the tree

builder.m1(attr1:1, attr2:2, { builder.m2(..., {...}) }

becomes equivalent to

builder.m1(attr1:1, attr2:2) { m2(...) {...} }

thanks to parens omission

43


Adding properties to numbers

Three possible approaches

• create a Category• a category is a kind of decorator for default MCs

• create a custom MetaClass• a full-blown MC class to implement and to set on the POGO instance

• use ExpandoMetaClass• friendlier DSL approach but with a catch

44


With a Category

class DistanceCategory { static Distance getMeters(Integer self) { new Distance(self, Unit.METERS) }}

use(DistanceCategory) { 100.meters}

Interesting scope: thread-bound & lexicalHave to surround with “use”• but there are ways to hide it

45


With an ExpandoMetaClass

Number.metaClass.getMeters = {-> new Distance(delegate, Unit.METERS) }

100.meters

Works for the class hierarchy for POJOs, and a flag exists to make it work for POGOs too

But the catch is it’s really a global change, so beware EMC enhancements collisions

46


Compile-time metaprogramming

Groovy 1.6 introduced AST Transformations

Compile-time == No runtime performance penalty!

Transformation47


AST Transformations

Two kinds of transformations

• Global transformations• applicable to all compilation units

• Local transformations• applicable to marked program elements• using specific marker annotations

48


Global transformations

Implement ASTTransformation

Annotate the transfo specifying a compilation phase

@GroovyASTTransformation(phase=CompilePhase.CONVERSION)public class MyTransformation implements ASTTransformation { public void visit(ASTNode[] nodes, SourceUnit unit) { ... }}

For discovery, create the file META-INF/services/org.codehaus.groovy.transform.ASTTransformation

Add the fully qualified name of the class in that file

49


Local transformations

Same approach as Globale transformationsBut you don’t need the META-INF fileInstead create an annotation to specify on which element the transformation should apply

@Retention(RetentionPolicy.SOURCE)@Target([ElementType.METHOD])@GroovyASTTransformationClass( ["fqn.MyTransformation"])public @interface WithLogging {...}

50


Example: the Spock framework

Changing the semantics of the original codeBut keeping a valid Groovy syntax

@Speckclass HelloSpock { def "can you figure out what I'm up to?"() { expect: name.size() == size

where: name << ["Kirk", "Spock", "Scotty"] size << [4, 5, 6] }}

Check out http://www.spockframework.org

51

http://www.spockframework.org

http://www.spockframework.org



Agenda

52


Various integration mechanisms

Java 6’s javax.script.* APIs (aka JSR-223)Spring’s language namespaceGroovy’s own mechanisms

But a key idea is to externalize those DSL programs• DSL programs can have their own lifecycle

• no need to redeploy an application because of a rule change• business people won’t see the technical code

53


Java 6’s javax.script.* API

Groovy 1.6 provides its own implementation of the javax.script.* API

ScriptEngineManager mgr = new ScriptEngineManager();ScriptEngine engine = mgr.getEngineByName(“Groovy”);

String result = (String)engine.eval(“2+3”);

54


Spring’s lang namespace

POGOs (Plain Old Groovy Objects) can be pre-compiled as any POJO and used interchangeably with POJOs in a Spring applicationBut Groovy scripts & classes can be loaded at runtime through the <lang:groovy/> namespace and tagReloadable on changeCustomizable through a custom MetaClass

<lang:groovy id="events" script-source="classpath:dsl/eventsChart.groovy" customizer-ref="eventsMetaClass" />

55


Groovy’s own mechanisms

Eval• for evaluating simple expressions

GroovyShell• for more complex scripts and DSLs

GroovyClassLoader• the most powerful mechanism

56


Eval

Simple mechanism to evaluate math-like formulas

Eval.me ( ‘3*4’)Eval.x (1, ‘3*x + 4’)Eval.xy (1, 2, ‘x + y’)Eval.xyz(1, 2, 3, ‘x * y - z’)

57


GroovyShell

A Binding provides a context of execution• can implement lazy evaluation if needed

A base script class can be specified

def binding = new Binding()binding.mass = 22.3binding.velocity = 10.6def shell = new GroovyShell(binding)shell.evaluate(“mass * velocity ** 2 / 2”)

58


GroovyClassLoader

Most powerful mechanism• could also visit or change the AST• scripts & classes can be loaded from elsewhere

• more control on compilation

GroovyClassLoader gcl = new GroovyClassLoader();Class clazz = gcl.parseClass( new File(“f.groovy”));GroovyObject instance = (GroovyObject)clazz.newInstance();instance.setMetaClass(customMC);

59


Externalize business rules

Although Groovy DSLs can be embedded in normal Groovy classes, you should externalize them

Store them elsewhere• in a database, an XML file, etc.

Benefits• Business rules are not entangled

in technical application code• Business rules can have their own lifecycle, without requiring

application redeployments

60



Agenda

61

Start small, with key concepts

Beware overengineering!

Grow your language progressively

Get your hands dirty

Play with the end-users

Let your DSL fly, it’s not yours, it’s theirs!

Tight feedback loop

Iterative process

Stay humble.

You can’t get it right the first time.

Don’t design alone at your deskInvolve the end users from the start

Playing it safein a sandbox


Various levels of sandboxing

Groovy supports the usual Java Security Managers

Use metaprogramming tricks to prevent calling / instanciating certain classes

Create a special GroovyClassLoader AST code visitor to filter only the nodes of the AST you want to keep

• ArithmeticShell in Groovy’s samples

69


Test, test, test!

Don’t just test for nominal cases• Explicitely test for errors!

Ensure end-users get meaninful error messages

70


Summary

Questions & Answers

Agenda

71


Summary

Groovy’s a great fit for Domain-Specific Languages• Malleable & flexible syntax• Full object-orientation

Metaprogramming capabilities• Runtime metaprogramming• Compile-time metaprogramming

Groovy’s very often used for mission-critical DSLs

72

I kan haz my cheezburgr naw?Or do ya reely haz keshtionz?

?


Appendix

74


http://www.flickr.com/photos/wheatfields/420088151/sizes/l/

http://www.flickr.com/photos/therefromhere/518053737/sizes/l/

http://www.flickr.com/photos/romainguy/230416692/sizes/l/

http://www.flickr.com/photos/addictive_picasso/2874279971/sizes/l/

http://www.flickr.com/photos/huangjiahui/3127634297/sizes/l/

http://www.flickr.com/photos/25831000@N08/3064515804/sizes/o/

http://www.flickr.com/photos/lanier67/3147696168/sizes/l/

http://www.flickr.com/photos/ktb/4916063/sizes/o/

http://www.flickr.com/photos/nathonline/918128338/sizes/l/

http://www.flickr.com/photos/kevinsteele/39300193/sizes/l/

http://commons.wikimedia.org/wiki/File:Brueghel-tower-of-babel.jpg

http://commons.wikimedia.org/wiki/File:Platypus.jpg

http://www.flickr.com/photos/joaomoura/2317171808/sizes/l/

http://www.flickr.com/photos/wiccked/132687067/

http://www.flickr.com/photos/timsamoff/252370986/sizes/l/

http://www.flickr.com/photos/29738009@N08/2975466425/sizes/l/

http://www.flickr.com/photos/howie_berlin/180121635/sizes/o/

http://www.flickr.com/photos/yogi/1281980605/sizes/l/

http://www.flickr.com/photos/dorseygraphics/1336468896/sizes/l/

http://www.flickr.com/photos/xcbiker/386876546/sizes/l/

http://www.flickr.com/photos/pietel/152403711/sizes/o/

75
























http://commons.wikimedia.org/wiki/File:Ornithorhynchidae-00.jpg

http://commons.wikimedia.org/wiki/File:Ornithorhynchidae-00.jpg

















http://www.flickr.com/photos/forezt/192554677/sizes/o/

http://keremkosaner.files.wordpress.com/2008/04/softwaredevelopment.gif

http://www.jouy.inra.fr

http://www.flickr.com/photos/ejpphoto/408101818/sizes/o/

http://www.flickr.com/photos/solaro/2127576608/sizes/l/

http://www.flickr.com/photos/biggreymare/2846899405/sizes/l/

76













Technology

Practical Groovy Domain-Specific Languages