Metaprogramming

CS251ProgrammingLanguagesSpring2018,LynTurbakDepartmentofComputerScienceWellesleyCollege

TheseslidesborrowheavilyfromBenWood’sFall‘15slides.

Howtoimplementaprogramminglanguage

InterpretaAonAninterpreterwriDenintheimplementaAonlanguagereadsaprogramwriDeninthesourcelanguageandevaluatesit.

TranslaAon(a.k.a.compilaAon)Antranslator(a.k.a.compiler)wriDenintheimplementaAonlanguagereadsaprogramwriDeninthesourcelanguageandtranslatesittoanequivalentprograminthetargetlanguage.

ButnowweneedimplementaAonsof:implementaAonlanguage

targetlanguageMetaprogramming 2

Metaprogramming:InterpretaAon

InterpreterforlanguageLonmachineM

MachineMPrograminlanguageL

Metaprogramming 3

Interpreters

Data

Output

Source Program

Interpreter =

virtual machine

Metaprogramming 4

Metaprogramming:TranslaAon

InterpreterforlanguageBonmachineM

MachineM

PrograminlanguageA AtoBtranslator

PrograminlanguageB

Metaprogramming 5

if (x == 0) { x = x + 1; } ...

cmp (1000), $0 bne L add (1000), $1 L: ...

C Source Program C Compiler

x86 Target Program

Compiler

x86 Target Program

x86 computer

Data

Output

Metaprogramming 6

ThankstoBenWoodfortheseandfollowingpictures

InterpretersvsCompilers

InterpretersNoworkaheadofLmeIncrementalmaybeinefficient

Compilers

AllworkaheadofLmeSeewholeprogram(ormoreofprogram)TimeandresourcesforanalysisandopLmizaLon

Metaprogramming 7

JavaCompiler

if (x == 0) { x = x + 1; } ...

load 0 ifne L load 0 inc store 0 L: ...

Source Program Java Compiler

Target Program

(compare compiled C to compiled Java) Metaprogramming 8

Compilers...whoseoutputisinterpreted

Target Program Java

Virtual Machine

Data

Output

Source Program Java Compiler

Target Program

Doesn’tthislookfamiliar? Metaprogramming 9

Interpreters...thatusecompilers.

Target Program

Virtual Machine

Data

Output

Source Program Compiler

Metaprogramming 10

JITCompilersandOpAmizaAon

Target Program

Virtual

Machine Data

Output

Just In Time Compiler

PerformanceMonitor

Source Program

Compiler

•  HotSpotJVM•  JikesRVM•  SpiderMonkey•  v8•  Transmeta•  ...

Metaprogramming 11

VirtualMachineModel

High-LevelLanguageProgram

VirtualMachineLanguage

Bytecodecompiler

Virtualmachine(interpreter)

JITcompiler

runAme

compileAme

Ahead-of-Lmecompiler

NaLveMachineLanguage

Metaprogramming 12

TypicalCompiler

Source Program

Lexical Analyzer

Syntax Analyzer

Semantic Analyzer

Intermediate Code Generator

Code Optimizer

Code Generator

Target Program

Analysis

Synthesis

Metaprogramming 13

Howtoimplementaprogramminglanguage

InterpreterRuleP-in-Lprogram Linterpretermachine

Pmachine

TranslatorRuleP-in-Sprogram S-to-Ttranslatormachine

P-in-Tprogram

Candescribebyderivinga“proof”oftheimplementaLonusingtheseinferencerules:

Metaprogramming 14

ImplementaAonDerivaAonExample

Provehowtoimplementa"251webpagemachine"using:

•  251-web-page-in-HTMLprogram(awebpagewriDeninHTML)•  HTML-interpreter-in-Cprogram(awebbrowserwriDeninC)•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)

Nopeakingahead!

Metaprogramming 15

ImplementaAonDerivaAonExampleSoluAon

Wecanomitsomeoccurrencesof“program”and“machine”:

Metaprogramming 16

ImplementaAonDerivaAonAreTreesAndsowecanrepresentthemasnestedstructures,likenestedbulletedlists:

251webpagemachine(I)q  251-web-page-in-HTMLprogramq  HTMLinterpretermachine(I)

²  HTML-interpreter-in-x86program(T)o  HTML-interpreter-in-Cprogramo  C-to-x86compilermachine(I)

•  C-to-x86compiler-in-x86program•  X86computer

²  x86computer

q  251-web-page-in-HTMLprogramo  HTML-interpreter-in-Cprogram

•  C-to-x86compiler-in-x86program•  X86computer

o  C-to-x86compilermachine(I)²  HTML-interpreter-in-x86program(T)²  x86computer

q  HTMLinterpretermachine(I)251webpagemachine(I)

Versionthatshowsconclusionsbelowbullets.MoresimilartoderivaLonswithhorizontallines,buthardertocreateandread

Preferred“top-down”versionthatshowsconclusionsabovebullets.

Metaprogramming 17

HowtoexecutetheRacketfactorialprogramgiventheseparts?

o  factorial-in-Racketprogramo  Racket-to-Python-translator-in-Pythonprogramo  Python-interpreter-in-Cprogramo  C-to-x86-translator-in-x86programo  x86computer(i.e.,x86interpretermachine)

DerivaAonExercise

Metaprogramming 18

Warning:cannotstartthefollowingway:

factorialmachine(I)q  factorial-in-Racketprogramq  Racketinterpretermachine(I)

….

Whynot?

DerivaAonExercise:SoluAonHowtoexecutetheRacketfactorialprogramgiventheseparts?

o  factorial-in-Racketprogramo  Racket-to-Python-translator-in-Pythonprogramo  Python-interpreter-in-Cprogramo  C-to-x86-translator-in-x86programo  x86computer(i.e.,x86interpretermachine)

Metaprogramming 19

PutyoursoluAonhere:

Metaprogramming:BootstrappingPuzzles

HowcanaJavacompilerbewriDeninJava?

HowcanaRacketinterpreterbewriDeninRacket?

Metaprogramming 20

Howcangcc(aC-to-x86compiler)bewriDeninC?

MetacircularityandBootstrappingManyexamples:•  LispinLisp/SchemeinScheme/RacketinRacket•  PythoninPython:PyPy•  JavainJava:JikesRVM,MaxineVM•  …•  C-to-x86compilerinC:gcc•  evalconstructinlanguageslikeLisp,JavaScript

Howcanthisbepossible?

Keyinsightstobootstrapping:•  ThefirstimplementaLonofalanguagecannotbeinitself,

butmustbeinsomeotherlanguage.•  OnceyouhaveoneimplementaLonofalanguageL,youcan

canimplement(enhancedversionsof)LinL.Metaprogramming 21

Supposeyouaregiven:•  Racket-interpreter-in-Pythonprogram•  Pythonmachine•  Racket-interpreter-in-RacketprogramHowdoyoucreateaRacketinterpretermachineusingtheRacket-interpreter-in-Racketprogram?

MetacircularityExample1:Problem

Metaprogramming 22

Supposeyouaregiven:•  Racket-interpreter-in-Pythonprogram•  Pythonmachine•  Racket-interpreter-in-RacketprogramHowdoyoucreateaRacketinterpretermachineusingtheRacket-interpreter-in-Racketprogram?

Racketinterpretermachine#2(I)q Racket-interpreter-in-Racketprogramq Racket-interpretermachine#1(I)

² Racket-interpreter-in-Pythonprogram² Pythonmachine

ButwhycreateRacketinterpretermachine#2whenyoualreadyhaveRacket-interpretermachine#1?

MetacircularityExample1:SoluAon

Metaprogramming 23

Supposeyouaregiven:•  Racket-subset-interpreter-in-Pythonprogram(implements

onlycoreRacketfeatures;nodesugaringorotherfrills)•  Pythonmachine•  Full-Racket-interpreter-in-Racket-subsetprogramHowdoyoucreateaFull-RacketinterpretermachineusingtheFull-Racket-interpreter-in-Racket-subsetprogram?

Full-Schemeinterpretermachine(I)q Full-Racket-interpreter-in-Racket-subsetprogramq Racket-subsetinterpretermachine#1(I)

² Racket-subset-interpreter-in-Pythonprogram² Pythonmachine

MetacircularityExample1:MoreRealisAc

Metaprogramming 24

Supposeyouaregiven:•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)•  C-to-x86-translator-in-Cprogram

HowdoyoucompiletheC-to-x86-translator-in-C?

MetacircularityExample2:Problem

Metaprogramming 25

Supposeyouaregiven:•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)•  C-to-x86-translator-in-Cprogram

HowdoyoucompiletheC-to-x86-translator-in-C?

MetacircularityExample2:SoluAon

C-to-x86-translatormachine#2(I)q C-to-x86-translator-in-x86program#2(T)

² C-to-x86-translator-in-C² C-to-x86-translatormachine#1(I)

o  C-to-x86-translator-in-x86program#1o  x86computer

q x86computer

ButwhycreateC-to-x86-translator-in-x86program#2(T)whenyoualreadyhaveC-to-x86-translator-in-x86program#1?

Metaprogramming 26

Supposeyouaregiven:•  C-subset-to-x86-translator-in-x86program

(acompilerforasubsetofCwriDeninx86)•  x86interpretermachine(anx86computer)•  Full-C-to-x86-translator-in-C-subsetprogram

(acompilerforthefullClanguagewriDeninasubsetofC)

HowdoyoucreateaFull-C-to-x86-translatormachine?

MetacircularityExample2:MoreRealisAc

Full-C-to-x86-translatormachine(I)q Full-C-to-x86-translator-in-x86program(T)

² Full-C-to-x86-translator-in-C-subset² C-subset-to-x86-translatormachine(I)

o  C-subset-to-x86-translator-in-x86programo  x86computer

q x86computerMetaprogramming 27

AlonglineofCcompilersC-version_n-to-target_n-translatormachine(I)q  C-version_n-to-target_n-translatorprogramintarget_n-1(T)

²  C-version_n-to-target_n-translatorprograminC-version_n-1²  C-version_n-1-to-target_n-1translatormachine(I)

o  C-version_n-1-to-target_n_1-translatorprogramintarget_n-2(T)

Ø  C-version_2-to-target_2-translator-programintarget_1(T)

§  C-version_2-to-target_2-translatorprograminC-version_1§  C-version_1-to-target_1translatormachine(I)

•  C-version_1-to-target_1-translatorprograminassembly_0•  assembly_0computer

Ø  target_1computer

o  target_n-2computerq  target_n-1computer

…

…

o  TheversionsofCandtargetlanguagescanchangeateachstage.o  Trojanhorsesfromearliersourcefilescanremainintranslatormachinesevenif

they’renotinlatersourcefile!SeeKenThompson’sReflec4ononTrus4ngTrustMetaprogramming 28

MoreMetaprogramminginSML

o  We’vealreadyseenPostFixands-expressionsinRacket;nextwe’llseehowtoimplementtheseinSML

o  TherestofthecourseexploresasequenceofexpressionlanguagesimplementedinSMLthatlookcloserandclosertoRacket:•  Intex:asimplearithmeLcexpressionlanguage•  Bindex:addnamingtoIntext•  Valex:addmorevaluetypes,dynamictypechecking,desugaringtoBindex

•  HOFL:addfirstclassfuncLonvalues,closurediagramstoValex

•  HOFLEC:addexplicitSML-likemutablecellstoHOFLMetaprogramming 29

Remember:language!=implementaAon

•  Easytoconfuse"thewaythislanguageisusuallyimplemented"or"theimplementaLonIuse"with"thelanguageitself.”

•  JavaandRacketcanbecompiledtox86

•  CcanbeinterpretedinRacket

•  x86canbecompiledtoJavaScript

•  CanwecompileC/C++toJavascript?hDp://kripken.github.io/emscripten-site/

Metaprogramming 30