Lecture 4: Compiler & Lambda Calculus

CSC 131!Fall, 2014!

!

Kim Bruce

Questions?

• Decidability?!

• LISP?

Compiler

• Translate from one VM to another!- Translate all units of program into object code!

- Link into single relocatable machine code!

- Load into memory!

- Begin execution

Lack of Purity

• Rarely pure compiler or interpreter!- Typically compile source into form easier to interpret. !

- Ex. Remove white space & comments, build symbol table, or parse each line and store in more compact form (e.g. tree)!

• Java originally hybrid!- Compile into JVML and then interpreted!

- Now compile to JVML & use just-in-time compiler on JVML

Compiler Structure

• Analysis:!- Break into lexical items, build parse tree, annotate

parse tree (e.g. via type checking)!

• Synthesis:!- generate simple intermediate code, optimization (look at instructions in context), code generation, linking and loading.

Symbol Table

• Symbol table: !- Contains all id names, !

- kind of id (vble, array name, proc name, formal parameter), !

- type of value, !

- where visible, etc.

Analysis

Source Program

Lexical Analysis

Symbol TableOther Tables

Lexical!Items

Syntax Analysis

Parse!Tree

Semantic Analysis

Annotated!Parse Tree

Synthesis

Symbol TableOther Tables

Inter.!Code

Optimiz-"ation

Optimized!Intermed.!

CodeCode

GenerationObject!Code

Inter. Code Generation

Annotated!Parse Tree

Portable Compilers

• Separate front and back-ends that share same intermediate code (GNU compilers)!- Write single front end for each language!

- Write single back end for each operating system - architecture combination.!

- Mix and match to build complete compilers!

• JVM starting to play that role too

Formal Syntax

Formal Syntax

• Syntax:!- Readable, writable, easy to translate, unambiguous, ... !

• Formal Grammars: !- Backus & Naur, Chomsky !

- First used in ALGOL 60 Report - formal description !

- Generative description of language. !

• Language is set of strings. (E.g. all legal C++ programs)

Example<exp> ⇒ <term> | <exp> <addop> <term>!

<term> ⇒ <factor> | <term> <multop> <factor>!

<factor> ⇒ <id> | <literal> | (<exp>) !

<id> ⇒ a | b | c | d !

<literal> ⇒ <digit> | <digit> <literal> !

<digit> ⇒ 0 | 1 | 2 | ... | 9 !

<addop> ⇒ + | - | or !

<multop> ⇒ * | / | div | mod | and

Extended BNF

• Extended BNF handy: !- item enclosed in square brackets is optional !

• <conditional> ⇒ if <expression> then <statement> [ else <statement> ] !

- item enclosed in curly brackets means zero or more occurrences !

• <literal> ⇒ <digit> <digit>

Syntax Diagrams

• Syntax diagrams - alternative to BNF. !- Syntax diagrams are never directly recursive, use

"loops" instead.

Ambiguity<statement> ⇒ <unconditional> | <conditional>!

<unconditional> ⇒ <assignment> | <for loop> | ! "" <statement> ""!! <conditional> ⇒ if (<expression>) <statement> |! if (<expression>) <statement> ! else <statement>!!How do you parse: ! if (exp1)! if (exp2)! stat1;! else! stat2;

Resolving Ambiguity

• Pascal, C, C++, and Java rule: !- else attached to nearest then.!

- to get other form, use ... !

• Modula-2 and Algol 68!- No ““, only “” (except write as “end”)!

• Not a problem in LISP/Racket/ML/Haskell conditional expressions!

• Ambiguity in general is undecidable

Chomsky Hierarchy

• Chomsky developed mathematical theory of programming languages: !- type 0: recursively enumerable !

- type 1: context-sensitive !

- type 2: context-free !

- type 3: regular !

• BNF = context-free, recognized by pda

Beyond Context-Free

• Not all aspects of PL’s are context-free!- Declare before use, goto target exist!

• Formal description of syntax allows: !- programmer to generate syntactically correct

programs !

- parser to recognize syntactically correct programs !

• Parser-generators: LEX, YACC, ANTLR, etc.!- formal spec of syntax allows automatic creation of

recognizers

Specifying Semantics: Lambda Calculus

Defining Functions

• In math and LISP:!- f(n) = n * n!

- (define (f n) (* n n))!

- (define f (lambda (n) (* n n)))!

• In lambda calculus!- λn. n * n!

- ((λn. n * n) 12) ⇒ 144

Pure Lambda Calculus

• Terms of pure lambda calculus!- M ::= v | (M M) | λv. M!

- Impure versions add constants, but not necessary!!

- Turing-complete!

• Left associative: M N P = (M N) P.!

• Computation based on substituting actual parameter for formal parameters

Free Variables

• Substitution easy to mess up!!

• Def: If M is a term, then FV(M), the collection of free variables of M, is defined as follows: !- FV(x) = x !

- FV(M N) = FV(M) ∪ FV(N) !

- FV(λv. M) = FV(M) - v

Substitution

• Write [N/x] M to denote result of replacing all free occurrences of x by N in expression M. !- [N/x] x = N, !

- [N/x] y = y, if y ≠ x, !

- [N/x] (L M) = ([N/x] L) ([N/x] M), !

- [N/x] (λy. M) = λy. ([N/x] M), if y≠ x and y ∉ FV(N), !

- [N/x] (λx. M) = λx. M.

Computation Rules

• Reduction rules for lambda calculus:! (α) λx. M → λy. ([y/x] M), if y ∉ FV(M).!

change name of parameters if new not capture old"

(β) (λx. M) N → [N/x] M. !computation by subst function argument for formal parameter"

(η) λx. (M x) → M. !Optional rule to get rid of excess λ’s

Why so complicated?

(λf. λz. f (f z)) (λx. x + z) → λz. (λx. x + z)((λx. x + z) z) !

→ λz. (λx. x + z)(z + z) !

→ λz. (z + z) + z = λz. 3z.!

- rather than the correct"

(λf. λy. f (f y)) (λx. x + z)) → λy. (λx. x + z)((λx. x + z) y) !

→ λy. (λx. x + z)(y + z) !

→ λy. (y + z) + z = λy. y + 2z.

illegal substitution

Normal Forms

• A term M is in normal form if no reduction rules apply, even after applications of α.!

• Not all terms have normal forms!- Ω = (λx. (x x))(λx. (x x))

How to evaluate

• Many strategies:!- (λx. x + 32)((λy. y * 3) 5) → (λx. x + 32) 15 → 47!

- versus!

- (λx. x + 32)((λy. y * 3) 5) → ((λy. y * 3) 5) + 32 → 47!

• Confluence: If M can be reduced to a normal form, then there is only one such normal form.!

• However, not all strategies give a normal form:!- (λx. 47) Ω

Inside-out

Outside-in

Computability

• Can encode all computable functions in pure untyped lambda calculus.!- true = λ u. λ v. u !

• true a b = a!

- false = λ u. λ v. v!

• false a b = b!

- cond = λ u. λ v. λ w. u v w!

• cond true a b = ? cond false a b = ?

Lambda Encoding

• Pairing:!- Pair = λ m. λ n. λ b. cond b m n. !

- fst = λp. p true !

• fst (Pair a b) = ?!

- snd = λp. p false !

• snd (Pair a b) = ?

Encoding Natural Numbers

• Natural numbers:!- 0 = λ s. λ z. z. !

- 1 = λ s. λ z. s z. !

- 2 = λ s. λ z. s (s z). !

• Integers encode repetition:!- 2 f x = f (f x)!

- 3 f x = f( f (f x))!

- n f x = f(n) (x)

Arithmetic

• Succ = λ n. λ s. λ z. s (n s z)!- Succ n = λ s. λ z. s (n s z) = λ s. λ z. s (s(n) z) = n+1!

• Plus = λ n. λ m. λ s. λ z. m s (n s z). !

• Mult = λ n. λ m. (m ( Plus n) 0). !

• isZero = λ n. n (λ x. false) true!

• Subtraction is hard!!

Predecessor

• PZero = <0,0> = Pair 0 0!

• PSucc = λ n. Pair (snd n) (Succ (snd n))!- PSucc PZero = <o,1>!

- n PSucc PZero = <n-1,n> for n > 0!

• Pred = λ n. fst (n PSucc PZero)!- Pred n = n - 1, for n > 0, !

- Pred 0 = 0

Recursion• Recursive definitions are handy!- fact = λn. cond (isZero n) 1 (Mult n (fact (Pred n)))!

- Not a legal definition in lambda calculus because can’t name functions!!

• Compute by expanding: !- fact 2 !

= cond (isZero 2) 1 (Mult 2 (fact (Pred 2))) !

= Mult 2 (fact 1)!

= Mult 2 (cond (isZero 1) 1 (Mult 1 (fact (Pred 1)))!

= Mult 2 (Mult 1 (fact 0)) = ... = Mult 2 (Mult 1 1) = 2

Recursion

• A different perspective: Start with!- fact = λn. cond (isZero n) 1 (Mult n (fact (Pred n)))!

• Let F stand for the closed term:!- λf. λn. cond (isZero n) 1 (Mult n (f (Pred n)))!

- Notice F(fact) = fact.!

- fact is a fixed point of F!

- To find fact, need only find fixed point of F!!

• Easy w/ g(x) = x * x, but F????

Fixed Points

• Several fixed point operators:!- Ex: Y = λf . (λx. f (xx))(λx. f (xx)) !

• Claim for all g, Y g = g (Y g)!Y g = (λf . (λx. f (xx))(λx. f (xx))) g !

= (λx. g(xx))(λx. g(xx)) !

= g((λx. g(xx)) (λx. g(xx))) !

= g (Y g)!

• If let x0 = Y g, then g (x0) = x0.

Invented by Haskell Curry

Factorial

• Recursive definition:!- let F = λf. λn. cond (isZero n) 1 (Mult n (f (Pred n)))!

- let fact = Y F!

- then F(fact) = fact because Y always gives fixed points!

• Compute:!fact 0 = (F (fact)) 0 because fact is a fixed point of F !

= cond (isZero 0) 1 (Mult 0 (fact (Pred 0))) !

= 1 by the definition of cond

Computing Factorials

fact 1 = (F (fact)) 1 because fact is a fixed point of F !

= (λn. cond (isZero n) 1 (Mult n (fact (Pred n)))) 1 expanding F!

= cond (isZero 1) 1 (Mult 1 (fact (Pred 1))) applying it"

= Mult 1 (fact (Pred 1)) by the definition of cond !

= fact 0 by the definition of Mult and Pred !

= 1 by the above calculation !

Lambda Calculus

• λ-calculus invented in 1928 by Church in Princeton & first published in 1932.!

• Goal to provide a foundation for logic!

• First to state explicit conversion rules.!

• Original version inconsistent, but corrected!- “If this sentence is true then 1 = 2” problematic!!!

• 1933, definition of natural numbers

Collaborators

• 1931-1934: Grad students: !- J. Barkley Rosser and Stephen Kleene!

- Church-Rosser confluence theorem ensured consistency (earlier version inconsistent)!

- Kleene showed λ-definable functions very rich!

• Equivalent to Herbrand-Gödel recursive functions!

• Equivalent to Turing-computable functions.!

• Founder of recursion theory, invented regular expressions!

• Church’s thesis: !- λ-definability ≡ effectively computable

OUndecidability

• Convertibility problem for λ-calculus undecidable.!

• Validity in first-order predicate logic undecidable.!

• Proved independently year later by Turing.!- First showed halting problem undecidable

Alan Turing• Turing!- 1936, in Cambridge, England, definition of Turing

machine!

- 1936-38, in Princeton to get Ph.D. under Church.!

- 1937, first published fixed point combinator!• (λx. λy. (y (x x y))) (λx. λy. (y (x x y)))!

- Kleene did not use fixed-point operator in defining functions on natural numbers!"

- Broke German enigma code in WW2, Turing test AI!

- Persecuted as homosexual, committed suicide in 1954

Combinatory Logic• Schönfinkel - get rid of bound variables!

• Combinators K, S sufficient to write all computable functions:!- K x y = x!

- S x y z = (x y)(x z)!

• CL terms include K, S, variables and all terms created by function application.!

• All computable fcns, equivalent to λ-calculus

Typed Lambda Calculus

Types

• Start with base type e and build up types and terms:!- Type ::= e | Type → Type!

- M ::= v | (M M) | λv : Type. M!

• Examples:!- Types: e, e → e, e → e → e, (e → e) → e, ...!

- Terms: λx: e. x, λf: e → e. λz: e. f(f(z))

Definitions

• Earlier definitions generalize over types t:!- truet = λx:t. λy:u. x!

- nt = λs: t→t. λz: t. s(n)(z)!

• Some untyped terms can’t be typed:!- Ω = (λx. (x x))(λx. (x x))!

- Y = λf . (λx. f (x x))(λx. f (x x))

Totally Awesome!!

• Theorem: If M is a term of the typed lambda calculus, then M has a unique normal form. I.e., every term of the typed lambda calculus is total.!

• Corollary: The typed lambda calculus does not include all computable functions.

Functional Languages

Commands/Statements

• Characteristics!- Support for variables !

• represent memory locations for storing updatable values. !

- Assignment operation !

• progress in computation depends on changes in values stored in variables. !

- Repetition!

• flow of control guided by conditional and looping statements controlling order in which assignment statements are executed.

Imperative Languages

• Organized around notion of commands!

• Meaning of a statement is operation which, based on current contents of memory, and explicit values supplied to it, modifies the current contents of memory.!

• How are results of one command communicated to next?

Problems

• Too low level.!

• Architecture dependent.!

• Does not generalize well to concurrent computation.

Expressions

• ... return a value, depending on the state of the computation.!

• Examples:!- Literals: 3, true, ”hello”, 42.56 !

- Aggregates: arrays, records, sets, lists, etc. E.g. [1,3,5] !

- Function calls: f(a,b), a + b * (c - d), (if x > 0 then sin else cos)(. . . ) !

- Conditional expressions: if x <> 0 then a/x else 1, case (historically only in functional languages) !

- Named constants and variables: pi, x

Declarative Language Test

• Within the scope of specific declarations of x1 , … , xn , all occurrences of an expression e containing only variables x1 , … , xn have the same value.!- Pure expressions are well behaved.!

• Fails w/imperative languages:!- x := x + y; y := 2 * x; !

- Two occurrences of x have different values.!

• If DLT holds, then allows for optimizations

Problems

• Order of evaluation!- if (i >= 0 && a[i] != 100) ...!

- What happens if i = -1?"

- Is && commutative?"

• Side effects!- Destroy nice properties!

- a[++x] = (y = x+1) + y + (++x)!

- Does 2*(x++) = (x++) + (x++)?

History of Functional Languages

• LISP 1957!

• Backus Turing award lecture (1977) - FP / FL!

• ML (1977) - Turing award for Robin Milner!- Meta language for theorem prover!

• SASL, KRC, and Miranda (Turner), Haskell (1990). !- Lazy languages!

- Implemented using combinatory logic

Functional Languages

• Pure LISP/Scheme (and Rex) are essentially sugared versions of untyped lambda calculus with constants.!

• ML & Haskell are typed lambda calculus w/type inference and recursion definitions at each type.

Haskell 98

• Purely functional!

• Functions are first-class values!

• Statically scoped!

• Strong, static typing via type inference !- Type-safe!

• Parametric polymorphism!

• Type classes

Haskell (cont)

• Rich type system including support for ADT’s !

• Non-strict (lazy) evaluation !

• Imperative features emulated using monads.!

• Garbage collection!

• Compiled or interpreted.!

• Named after Haskell Curry -- early contributor to lambda calculus and combinatory logic

Read Haskell Tutorials

• All on links page from course web page!

• I like “Learn you a Haskell for greater good”!

• O’Reilly text: “Real World Haskell” free on-line!

• Print Haskell cheat sheet!

• Glasgow Haskell compiler, available at !- http://www.haskell.org/ghc/

Using GHC

• to enter interactive mode type: ghci !- :load myfile.hs -- :l also works!

- after changes type :reload myfile.hs!

- Control-d to exit!

- :set +t -- prints more type info when interactive!

- “it” is result of expression

Built-in data types• Unit has only ( )!

• Bool: True, False with not, &&, ||!

• Int: 5, -5, with +, -, *, ^, =, /=, <, >, >=, ...!- div, mod defined as prefix operators (`div` infix)!

- Int fixed size (usually 64 bits)!

- Integer gives unbounded size!

• Float, Double: 3.17, 2.4e17 w/ +, -, *, /, =, <, >, >=, <=, sin, cos, log, exp, sqrt, sin, atan.

More Basic Types

• Char: ‘n’!

• String = [Char], not really primitive!- "hello"++" there", length!

- No substring, but `isInfixOf` for all lists!

- Also ‘isPrefixOf`, `isSuffixOf ’!

• Type classes (later) provide relations between classes.

Prefix op w/out ``!

import Data.List

list of Char

Interactive Programming with ghci

• Type expressions and will evaluate!

• Define abbreviations with “let”!- let double n = n + n!

- let seven = 7!

• “let” not necessary at top level in programs loaded from files

Lists

• Lists!- [2,3,4,9,12]: [Integer]!

- [] -- empty list!

- Must be homogenous!

- Functions: length, ++, :, map, rev !

• also head, tail, but normally don’t use!

Polymorphic Types

• [1,2,3]:: [Integer]!

• [“abc”, “def”]:: [[Char]], ...!

• []:: [a]!

• map:: (a → b) → ([a] → [b])!

• Use :t exp to get type of exp

Pattern Matching

• Decompose lists:!- [1,2,3] = 1:(2:(3:[]))!

• Define functions by cases using pattern matching:!

prod [] = 1 prod (fst:rest) = fst * (prod rest)

Pattern Matching

• Desugared through case expressions:!- head' :: [a] -> a

head' [] = error "No head for empty lists!" head' (x:_) = x !

• equivalent to !- head' xs = case xs of

[] -> error "No head for empty lists!" (x:_) -> x

Type constructors

• Tuples!- (17,”abc”, True) : (Integer , [Char] , Bool)!

- fst, snd defined only on pairs!

• Records exist as well

More Pattern Matching

• (x,y) = (5 `div` 2, 5 `mod` 2)!

• hd:tl = [1,2,3]!

• hd:_ = [4,5,6]!- “_” is wildcard.

Static Typing

• Strongly typed via type inference!- head:: [a] → a

tail:: [a] → [a]!

- last [x] = xlast (hd:tail) = last tail!

• System deduces most general type, [a] -> a!- Look at algorithm later

Static Scoping

• What is the answer?!- let x = 3!- let g y = x + y!- g 2!- let x = 6!- g 2!

• What is the answer in original LISP?!- (define x 3)!- (define (g y) (+ x y))!- (g 2)!- (define x 6)!- (g 2)

Questions?