Upload
wim-vanderbauwhede
View
248
Download
2
Embed Size (px)
DESCRIPTION
These are the slides of the talk I gave at the Dyla'14 workshop (http://conferences.inf.ed.ac.uk/pldi2014/). It's about monads for languages like Perl, Ruby and LiveScript. The source code is available at https://github.com/wimvanderbauwhede/Perl-Parser-Combinators https://github.com/wimvanderbauwhede/parser-combinators-ls Don't be put off by the word monad or the maths. This is basically a very practical way for doing tasks such as parsing.
Citation preview
List-based Monadic Computations forDynamically Typed LanguagesWim VanderbauwhedeSchool of Computing Science, University of Glasgow, UK
OverviewI Program Composition in Dynamically
Typed LanguagesI ObjectsI Monads
I Defining KarmaI Karmic TypesI Combining Functions using FunctionsI The Rules
I Designing KarmaI Syntactic SugarI fold and chainI Karma Needs Dynamic Typing
I Karmic Computations By ExampleI MaybeI StateI Parsing
I Conclusion
Dynamically Typed Languages
size proportional to√
#projects on GitHub
Perl ...
I Notorious
$_=’while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map{$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110;$t^=(72,@z=(64,72,$a^=12*($_%16 -2?0:$m&17)),$b^=$_%64?12:0,@z)[$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h =5;$_=unxb24,join"",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$ d=unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d>>12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q*8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]}print+x"C*",@a}’;s/x/pack+/g;eval;
I But I like Perl (*ducks*)
Program Composition
I Most Dynamic Languages use an OOP approach tostructure/compose programs
I But what if you don’t like object orientation?I Or what if it is simply not the best approach?I An alternative approach to program composition:
I Functional Languages use composition of higher-order functionsI In particular, Haskell uses monads ...
Haskell ...
I Most empathically notdynamically typed ...
I Makes easy things hard ...I But I like Haskell (*ducks*)
Monads
a reputation of being esoterichard to understandthe province of academiclanguages(unknown poet, 21st century)
Monas Hieroglyphica,(“The Hieroglyphic Monad”),
John Dee, Antwerp 1564
Monads in Dynamic Languages
I Have been done many times for many languageshttps://en.wikipedia.org/wiki/Monad(functional_programming)#Monads_in_other_languages
I But have not found widespread adoptionI So ...
The Claim
I ... are monads of practical use in day-to-day programming?I Yes!I Only we won’t call them monads ...
Karmic Computations
I Karma is a Sanskrit word meaning action, work or deed.I A different approach from the many existing implementations of
monads in dynamic languages.I Based on types, lambda functions and higher-order functions.
Types and NotationI Haskell-style syntax for types:
I A function g from a type a to a type b:
g :: a→ b
I A function s with two arguments and the result of type a:
s :: a→ a→ a
I A function f from a type a to a type m b, i.e. a type m parametrisedon a type b:
f :: a→ m b
I A function h which takes as arguments two functions of typea→ b and b → c and returns a function of type a→ m b:
h :: (a→ b)→ (b → c)→ (a→ m b)
Lambda Functions
I Also knows as anonymous subroutines, i.e. functions without aname.
I They are expressions that can be assigned to variables.
Language Lambda SyntaxHaskell \x y -> f x y
LiveScript (x,y) -> f x,y
Perl sub ($x,$y) { f $x,$y }
Python lambda x,y : f(x,y)
Ruby -> (x,y) { f x,y }
Higher-Order Functions
I Functions that take otherfunctions as arguments and/orreturn functions as results
I Available in your favouritedynamically type language: Ruby,Python, LiveScript, JavaScript, ...
What is Karma?
A karmic computation consists of:I a given type m a;I the higher-order functions bind and wrapI and three rules governing those functions.
Karmic Types
I A karmic type can be viewed as a “wrapper” around another type.I It does not have to have a particular structure, e.g. it can be an
object, or a list, or a map, or a function.I The key point is that the “karmic” type contains more information
than the “bare” type.I We will write m a as the karmic type constructor, regardless of
the structure of the type.I For example, in Ruby the following function f has a type
f :: a→ m b, if g has type g :: a→ b:
def f(x)m.new( g(x) )
end
Combining Functionsusing Functions
The purpose of the karmic computation is to combine functions thatoperates on karmic types.
I The bind function composes values of type m a with functionsfrom a to m b:bind :: m a → (a → m b)→ m b
I The wrap function takes a given type a and returns m a:wrap :: a → m a
With these two functions we can create expressions that consists ofcombinations of functions.
The Rules
Three rules govern bind and wrap to ensure that the composition iswell-behaved:
1. bind (wrap x) f ≡ f x2. bind m wrap ≡ m3. bind (bind m f) g ≡ bind m (\x -> bind (f x) g)
For reference, the rules in Ruby syntax:
1. bind(wrap(x),f) = f.call(x)2. bind(m,wrap) = m3. bind(bind(m,f),g) = bind(m,->(c){bind(f.call(x),g))})
Syntactic Sugar for MonadsIn Haskell:
I No sugar:
ex x =\x -> (bind (g x) (\y -> (f (bind y (\z -> wrap z)))))
I Operator sugar:
ex x =g x >>= \y ->f y >>= \z ->wrap z
I Do-notation sugar:
ex x = doy <- g xz <- f ywrap z
Syntactic Sugar for Karma
Our approach:I Use lists to combine computationsI For example:
[word,maybe parens choicenatural,[symbol ’kind’,symbol ’=’,natural
]]
Designing Karma
I Grouping items in lists provides a natural sequencingmechanism, and allows easy nesting.
I In most dynamic languages, lists us the [...,...] syntax, it’sportable and non-obtrusive.
I We introduce a chain function to chain computations together.I Pass chain a list of functions and it will combine these functions
into a single expression.
Preliminary: fold
I To define chain in terms of bind and wrap, we need the left foldfunction (foldl).
I foldl performs a left-to-right reduction of a list via iterativeapplication of a function to an initial value:foldl :: (a→ b → a)→ a→ [b]→ [a]
I A possible implementation e.g. in Perl would be
sub foldl ($f, $acc, @lst) {for my $elt (@lst) {$acc=$f->($acc,$elt)
}return $acc
}
Defining chain
I We can now define chain as
sub chain(@fs) {sub($x) { foldl bind, wrap $x, @fs }
}
I Or in Haskell syntax
chain fs = \x -> foldl (>>=) (wrap x) fs
Karma Needs Dynamic Typing
I The above definition of chain is valid in Haskell, but only if thetype signature ischain :: [(a→ m a)]→ (a→ m a)
I But the type of bind is bind :: m a → (a → m b)→ m bI the karmic type m b returned by bind does not have to be the same
as the karmic type argument m a.I every function in the list passed to chain should be allowed to have
a different type!
I This is not allowed in a statically typed language like Haskell, soonly the restricted case of functions of type a→ m a can beused.
I However, in a dynamically typed language there is no suchrestriction, so we can generalise the type signature:chain :: [(ai−1 → m ai )]→ (a0 → m aN) ∀i ∈ 1 ..N
Demonstrating Karma
I Combining computations that can failI Stateful computationI Parser combinators
Combining Fallible Computations (1)
I Expressing failureI A Maybe class
class Maybeattr_accessor :val,:okdef initialize(value,status)@val=value@ok=status # true or false
endend
I For convenience: a fail instance
fail = Maybe.new(nil,false)
I Assume f ,g,h :: a→ Maybe b
Combining Fallible Computations (2)
I Without karma
v1 = f(x)if (v1.ok)v2=g(v1.val)if (v2.ok)v3 = h(v2.val)if (v3.ok)v3.val
elsefail
endelsefail
endelsefail
end
I With karma:
comp = chain [->(x) {f x},->(y) {g y},->(z) {h z}]
v1=comp.call x
I If f, g and h are defined aslambda functions:
comp =chain [ f,g,h ]
v1=comp.call x
bind and wrap for Maybe
I For the Maybe karmic type, bind is defined as:
def bind(x,f)if x.ok
f.call(x)else
failend
end
I and wrap as:
def wrap(x)Maybe.new(x,true)
end
Stateful Computation
I Using karma to perform stateful computations without actualmutable state.
I Example in LiveScript, because LiveScript has the option to forceimmutable variables.
I There is a growing interest in the use of immutable state in otherdynamic languages, in particular for web programming, becauseimmutable state greatly simplifies concurrent programming andimproves testability of code.
State Concept and API
I We simulate state by combining functions that transform thestate, and applying the resulting function to an initial state(approach borrowed from Haskell’s Control.State).
I A small API to allow the functions in the list to access a sharedstate:
get = -> ( (s) -> [s, s] )put = (s) -> ( -> [[],s])
The get function reads the state, the put function writes anupdated state. Both return lambda functions.
I A top-level function to apply the stateful computation to an initialstate:
res = evalState comp, init
whereevalState = (comp,init) -> comp!(init)
Trivial Interpreter:Instructions
I We want to interpret a list of instructions of typetype Instr = (String, ([Int ]→ Int , [a]))
I For exampleinstrs = [
["v1",[set,[1]]],["v2",[add,["v1",2]]],["v2",[mul,["v2","v2"]]],["v1",[add,["v1","v2"]]],["v3",[mul,["v1","v2"]]]
]
where add, mul and set are functions of type [Int ]→ IntThe interpreter should return the final values for all variables.
I The interpreter will need a context for the variables, we use asimple map { String : Int } to store the values for each variable.This constitutes the state of the computation.
Trivial Interpreter:the Karmic Function
I We write a function interpret_instr_st :: Instr → [[a],Ctxt ]:
interpret_instr_st = (instr) ->chain( [
get,(ctxt) ->
[v,res] = interpret_instr(instr, ctxt)put (insert v, res, ctxt)
] )
I interpret_instr_stI gets the context from the state,I uses it to compute the instruction,I updates the context using the insert function andI puts the context back in the state.
Trivial Interpreter:the Stateless Function
I Using Ctxt as the type of the context, the type of interpret_instr isinterpret_instr :: Instr → Ctxt → (String, Int) and theimplementation is straightforward:
interpret_instr = (instr, ctxt) ->[v,rhs] = instr # unpacking[op,args] = rhs # unpackingvals = map (‘get_val‘ ctxt), args[v, op vals]
I get_val looks up the value of a variable in the context; it isbackticked into an operator here to allow partial application.
bind and wrap for State
I For the State karmic type, bind is defined as:
bind = (f,g) ->(st) ->[x,st_] = f st(g x) st_
I and wrap as
wrap = (x) -> ( (s) -> [x,s] )
modify: Modifying the State
I The pattern of calling get to get the state, then computing usingthe state and then updating the state using put is very common.
I So we combine it in a function called modify:modify = (f) ->
chain( [get,(s) -> put( f(s))
] )
I Using modify we can rewrite interpret_instr_st asinterpret_instr_st = (instr) ->
modify (ctxt) ->[v,res] = interpret_instr(instr, ctxt)insert v, res, ctxt
mapM: a Dynamic chain
I We could in principle use chain to write a list of computations oneach instruction:
chain [interpret_instr_st instrs[0],interpret_instr_st instrs[1],interpret_instr_st instrs[2],...
]
but clearly this is not practical as it is entirely static.I Instead, we use a monadic variant of the map function, mapM:
mapM :: Monad m⇒ (a→ m b)→ [a]→ m [b]mapM applies the computations to the list of instructions (usingmap) and then folds the resulting list using bind.
I With mapM, we can simply write :
res = evalState (mapM interpret_instr_st, instrs), {}
Fortran ...
I I like Fortran (*ducks*)
Parser Combinators
I Or more precisely, I needed toparse Fortran for sourcetransformations.
I So I needed a Fortran parser ...I ... in Perl (for reasons).I So I created a Perl version of
Haskell’s Parsec parsercombinator library ...
I ... using Karma.
Parser Combinator BasicsI Each basic parser returns a function which operates on a string
and returns a result.
Parser :: String → Result
I The result is a tuple containing the status, the remaining stringand the substring(s) matched by the parser.
type Result = (Status,String, [Match])
I To create a complete parser, the basic parsers are combinedusing combinators.
I A combinator takes a list of parsers and returns a parsingfunction similar to the basic parsers:
combinator :: [Parser ]→ Parser
Karmic Parser Combinators
I For example, a parser for a Fortran-95 type declaration:
my $parser = chain [identifier,maybe parens choicenatural,[
symbol ’kind’,symbol ’=’,natural
]];
I In dynamic languages the actual chain combinator can beimplicit, i.e. a bare list will result in calling of chain on the list.
I For statically typed languages this is of course not possible as allelements in a list must have the same type.
bind and wrap for Parser
I For the Parser karmic type, bind is defined as (Perl):
sub bind($t, $p) {my ($st_,$r_,$ms_) = @{$t};if ($st_) {($st,$r,$ms) = $p->($r_);if ($st) {(1,$r,[@{$ms_},@{$ms}]);
} else {(0,$r,undef)
}} else {(0,$r_,undef)
}
I and wrap as:
sub wrap($str){ (1,$str,undef) }
bind and wrap for Parser
I For the Parser karmic type, bind is defined as (LiveScript):
bind = (t, p) ->[st_,r_,ms_] = tif st_ then[st,r,ms] = p r_if st then[1,r,ms_ ++ ms]
else[0,r,void]
else[0,r_,void]
I and wrap as:
wrap = (str) -> [1,str,void]
Final Example (1)
I Parsers can combine several other parsers, and can be labeled:
my $f95_arg_decl_parser =chain [whiteSpace,{TypeTup => $type_parser},maybe [
comma,$dim_parser
],maybe [
comma,$intent_parser
],symbol ’::’,{Vars => sepBy ’,’,word}
];
Final Example (2)
I We can run this parser e.g.
my $var_decl ="real(8), dimension(0:ip,-1:jp+1,kp) :: u,v"my $res =run $f95_arg_decl_parser, $var_decl;
I This results in the parse tree:
{’TypeTup’ => {
’Type’ => ’real’,’Kind’ => ’8’
},’Dim’ => [’0:ip’,’-1:jp+1’,’kp’],’Vars’ => [’u’,’v’]}
Parser Combinators Library
I The Perl and LiveScript versions are on GitHub:I https://github.com/wimvanderbauwhede/Perl-Parser-Combinators
I https://github.com/wimvanderbauwhede/parser-combinators-ls
I The Perl library is actually used in a Fortran source compilerI https://github.com/wimvanderbauwhede/RefactorF4Acc
Conclusion
I We have presented a design for list-based monadiccomputations, which we call karmic computations.
I Karmic computationsI are equivalent to monadic computations in statically typed
languages such as Haskell,I but rely essentially on dynamic typing through the use of
heterogeneous lists.
I The proposed list-based syntaxI is easy to use,I results in concise, elegant and very readable code, andI is largely language-independent.
I The proposed approach can be applied very widely, especially asa design technique for libraries.
Thank you