View
218
Download
0
Category
Tags:
Preview:
Citation preview
Design and Implementation of Object Oriented Dynamic Programming Languages
Jonathan Bachrach
Greg Sullivan
Kostas Arkoudous
Who am I?
• Jonathan Bachrach• PhD in CS: neural networks applied to
robotic control• Experience in real-time high performance
electronic music• Involved in design of Sather and Dylan• Five years experience writing Dylan’s
runtime and compiler at Harlequin
The Seminar
• 6.894, 26-311, 1-2.30, 3-0-9, 3 EDP’s• Theme• Proto running example• Lectures• Readings + Presentations• Panels• Assignments• Projects
Today
• Taste of the Seminar
• Evolutionary Programming
• Proto
• Language Design in the New Millenium
• Language Implementation Techniques
• Seminar Details
Language Renaissance
• Perl• Python• MetaHTML• PHP• TCL• Cecil• C#• Java
• JavaScript• Sather• Rebol• Curl• Dylan• Isis• Limbo• …
The Stage
• Increasing Demands for Quick Time to Market• Moore’s Law for Hardware but is software
keeping up?• Most Time Spent after initial deployment• Complex environments
– Distributed– Agents– Real world– Continuous, non-deterministic, dynamic inputs
Conventional Programming
• Assume user knows exactly what they want before programming
• Assume that specifications don’t change over time
• Problem: Forces premature decisions and implementation effort
• Example: Java Class Centric Methods
Evolutionary Programming
• Need to play with prototypes before knowing what you really want– Support rapid prototyping– Smooth transition to delivery
• Requirements change all the time– Late binding allowing both developers and
program alike to update behavior
Language Design:User Oriented Goals
• Perlis: “A language that doesn't affect the way you think about programming, is not worth knowing.”
• What user-oriented goals would you suggest would result in the best language?
Proto Goals
• Simple• Productive• Powerful• Extensible• Dynamic• Efficient• Real-time
• Teaching and research vehicle
• Electronic music is domain to keep it honest
Proto Ancestors
• Language Design is Difficult– Leverage proven ideas– Make progress in selective directions
• Ancestors– Scheme– Cecil– Dylan
Proto <=> Scheme
• Concise naming• Procedural macros• Objects all the way
• Long-winded naming• Rewrite rule only• Only records
Proto <=> Cecil
• Prefix syntax• Scheme inspired
special forms
• Infix syntax• Smalltalk inspired
special forms
Proto <=> Dylan
• Prefix syntax• Prototype-based• Procedural macros• Rationalized collection
protocol / hierarchy• Always open• Predicate types
• Infix syntax• Class-based• Rewrite-rule only …• Conflated collection
protocol / hierarchy• Sealing• Fixed set of types
Prototype-based OO
• Simplified object model
• No classes
• Cloning basic operation for instance and prototype creation
• Prototypes are special objects that serve as classes
• Inheritance follows cloned-from relation
Proto: OO & MM
(dv <point> (isa <any>))
(slot <point> (point-x <int>) 0)
(slot <point> (point-y <int>) 0)
(dv p1 (isa <point>))
(dm + ((p1 <point>) (p2 <point>) => <point>)
(isa <point>
(set point-x (+ (point-x p1) (point-x p2)))
(set point-y (+ (point-y p1) (point-y p2))))
Language Design:User Goals -- The “ilities”
• Learnability
• Understandability
• Writability
• Modifiability
• Runnability
• Interoperability
Learnability
• Simple
• Small
• Regular
• Gentle learning curve
• Perlis: “Symmetry is a complexity reducing concept…; seek it everywhere.”
Proto: Learnability
• Simple and Small: – 16 special forms: if, seq, set, fun, let, loc, lab,
fin, dv, dm, dg, isa, slot, ds, ct, quote
– 7 macros: try, rep, mif, and, or, select, case
• Gentle Learning Curve: – Graceful transition from functional to object-oriented
programming
– Perlis: “Purely applicative languages are poorly applicable.”
Proto: Special Forms
IF (IF ,test ,then ,else)SEQ (SEQ ,@forms)SET (SET ,name ,form) | (SET (,name ,@args) ,form)LET (LET ((,var ,init) …) ,@body)FUN (FUN ,sig ,@body)LOC (LOC ((,name ,sig ,@body) …) .@body)LAB (LAB ,name ,@body)FIN (FIN ,protected-form ,@cleanup-forms)DV (DV ,var ,form)DM (DM ,name ,sig ,@body)DG (DG ,name ,sig)ISA (ISA (,@parents) ,@slot-inits)SLOT (SLOT ,owner ,var ,init)
sig (,@vars) | (,@vars => ,var)var ,name | (,name ,type)slot-init (SET ,name ,value)
Understandability
• Natural notation
• Simple to predict behavior
• Modular
• Models application domain
• Concise
Proto: Understandability
• Describable by a small interpreter– Size of interpreter is a measure of complexity
of language
• Regular syntax– Debatable whether prefix is natural, but it’s
simple, regular and easy to implement
Writability
• Expressive features and abstraction mechanisms
• Concise notation
• Domain-specific features and support
• No error-prone features
• Internal correctness checks (e.g., typechecking) to avoid errors
Tradeoff One: Abstraction <=> Writability
• Abstraction can obscure code– Example: abuse of macros
• Concision can obscure code– Example: APL
Tradeoff Two:Domain Specific <=> Simplicity
• Challenge is to introduce simple domain specific features that don’t hair up language– CL has reader macros for introducing new
token types
Proto: Error Proneness
• No out of language errors– At worst all errors will be be caught in
language at runtime– At best potential errors such as “no applicable
methods” will be caught statically earlier and in batch
• Unbiased dispatching and inheritance– Example: Method selection not based on
lexicographical order as in CLOS
Design Principle Two:Planned Serendipity
• Serendipity:– M-W: the faculty or phenomenon of finding
valuable or agreeable things not sought for
• Orthogonality– Collection of few independent powerful
features combinable without restriction
• Consistency
Proto: Serendipity
• Objects all the way down• Slots accessed only through calls to
generic’s• Simple orthogonal special forms• Expression oriented• Example:
– Exception handling can be built out of a few special forms: lab, fin, loc, …
Modifiability
• Minimal redundancy
• Hooks for extensibility included automatically
• No features that make it hard to change code later
Proto: Extensible Syntax
• Syntactic Abstraction
• Procedural macros
• WSYWIG– Pattern matching– Code generation
• Example:(ds (unless ,test ,@body)
`(if (not ,test) (seq ,@body)))
Proto: Multimethods
• Can add methods outside original class definition:– (dm jb-print ((x <node>)) …)– (dm jb-print ((x <str>)) …)
Proto: Generic Accessors
• All slot access goes through generic function calls
• Can easily redefine these generic’s without affecting client code
Runnability
• Features for programmers to control efficiency
• Analyzable by compilers and other tools
• Note: this will be a running theme!
Tradeoff Three:Runnability <=> Simplicity
• Much of a language design can be dedicated to providing mechanisms to control efficiency (e.g., sealing in Dylan)
• Obscure algorithms
• Perlis: “A programming language is low level when its programs require attention to the irrelevant.”
Proto: Optional Types
• All bindings and parameters can take optional types
• Rapid prototype without types
• Add types for documentation and efficiency
• Example:(dm format (s msg (args …)) …)
(dm format ((s <stream>)(msg <str>) (args …)) …)
Proto: Pay as You Go
• Don’t charge for features not used• Pay more for features used in more complicated
ways• Examples:
– Dispatch• Just function call if method unambiguous from argument types• Otherwise require dynamic method lookup
– Proto’s bind-exit called “lab”• Local exits are set + goto• Non local exits must create a frame and stack alloc an exit
closure
Interoperability
• Portability
• Foreign Language Interface
• Directly or indirectly after mindshare
The Rub
• Support for evolutionary programming creates a serious challenge for implementors
• Straightforward implementations would exact a tremendous performance penalty
The Game
• Every Problem in CS can be Solved with another Level of Indirection -- ancient proverb
• Every Optimization Problem can be Viewed as Removing a Level of Indirection -- jb
• Example: Procedures <=> Inlining
Implementation Techniques
• System code techniques: – Often called runtime optimizations– Turbo charges user code– Example: caching
• User code rewriting techniques: – Often called compile-time optimizations– Example: inlining
Implementing Prototypes
• Problem– No classes only objects– But objects need descriptions of slots and state– Would be too expensive for each object to have a copy
of these descriptions
• A solution:– Create classes called traits on demand
• When you are cloned or• When slots are added to you
– Objects contain their values and share traits through a traits pointer
Proto: Traits on Demand
x: 89
y: 55
fo lks: x:
x: 33
y: 67
fo lks: x:
x: 0
y: 0
fo lks: x:
89
55
x:
33
67
x:
x: 0
y: 0
fo lks: x:
0
0
x:
< po in t>
p1 p2
p1 p2
< po in t>
< po in t>T ra its
Implementing Generic Dispatch
• Multimethod dispatch– Considers all arguments
– Orders methods based on specializer type specificity
• Naïve description:– Find applicable methods
– Sort applicable methods
– Call most specific method
• Problem: too expensive
Dispatch Technique One:Dispatch Cache
• Hierarchical cache – Each level discriminates one argument using
associative mechanism– Keys are concrete object-traits– Values are either
• Cache for remaining arguments or• Method to call with given arguments
• Problems– Too many indirections:
• generic call + method call• key and value lookups
Engine Node Dispatch
• Glenn Burke and myself at Harlequin, Inc.
circa 1996-– Partial Dispatch: Optimizing Dynamically-Dispatched Multimethod Calls
with Compile-Time Types and Runtime Feedback, 1998
• Shared decision tree built out of executable engine nodes
• Incrementally grows trees on demand upon miss
• Engine nodes are executed to perform some action typically tail calling another engine node eventually tail calling chosen method
• Appropriate engine nodes can be utilized to handle monomorphic, polymorphic, and megamorphic discrimination cases corresponding to single, linear, and table lookup
Engine Node Dispatch Picture
textcall
\+<i>,<i>m ethod
NAM
\+<f>,<f>m ethod
...discrim inator
...
<generic>
...
<f>
<i>
linear ep
<linear-engine>
...
< i>
m ono ep
<mono-engine>
...
<f>
m ono ep
<mono-engine>
...MEP
<method>
...MEP
<method>
...MEP
<method>
Define method \+ (x :: <i>, y :: <i>) … end;Define method \+ (x :: <f>, y :: <f>) … end;Seen (<i>, <i>) and (<f>, <f>) as inputs.
Dispatch Technique Two:Decision Tree
• Intelligently number traits with id’s– Children tend to be in contiguous id range
• Label all traits according to most applicable method• Construct decision tree constructing largest class id
ranges by merging• Implement with simple numeric operations and JIT
code generation• Problem:
– Generic call + method call– Lost inlining opportunities
Code Rewriting Technique One:Inlining Dispatch
• Inline when number of methods is small• When methods themselves are small• Example: List operations• Twist: Partially inline dispatch when there is a
spike in the method frequencies• Example: Numeric operations• Problem: Lose downstream type inference
possibilities because result of call gets merged back into all other possibilities
Inlining Dispatch Example One
(dm empty? ((x <lst>)) #f)
(dm empty? ((x nil)) #t)
(dm null? ((x <lst>))
(empty? X))
=>
(dm null? ((x <lst>))
(if (== x nil) #t
(if (isa? X <lst>) #f
(error …))))
Inlining Dispatch Example Two
(dm sum+1 (x y)
(+ (+ x y) 1))
=>
(dm sum+1 (x y)
(let ((t (if (and (isa? x <int>) (isa? y <int>))
(i+ x y)
(+ x y))))
(if (isa? t <int>)
(i+ t 1)
(+ t 1))))
Code Rewriting Technique Two:Code Splitting
• Clone code below typecase so each branch can run with tighter types
• Problem: potential code explosion
Code Splitting Example
(dm sum+1 (x y)
(+ (+ x y) 1))
=>
(dm sum+1 (x y)
(if (and (isa? x <int>) (isa? y <int>))
(i+ (i+ x y) 1)
(+ (+ x y) 1)))
Summary
• Touched on evolutionary programming
• Introduced Proto
• Discussed language design
• Sketched out several implementation techniques to gain back performance
Today’s Reading List
• Dijkstra: Goto harmful
• Hoare: Hints on PL design
• Steele: Growing a Language
• Gabriel: Worse is Better
• Chambers: Synergies Between Object-Oriented Programming Language Design and Implementation Research
• Chambers: The Cecil Language
• Norvig: Adaptive Software
• Cook: Interfaces and Specifications for the Smalltalk-80 Collection Classes
• XP: www.extremeprogramming.com
• *Lee: Advanced Language Implementation
• *Queinnec: Lisp In Small Pieces
Open Problems
• Extensible enough language to stem need for domain specific languages
• Best of both worlds of performance and dynamism
• Simplest combination of language + implementation
• …
Course
• Seminar style format• Encourage discussion• Identify and make progress towards the
solution of open problems• Course mostly not about language design
– Will use proto as a point of discussion– Will talk about language design implications
along the way
Prerequisites
• 6.001 (SICP)
• 6.035 (Computer Language Engineering)
• 6.821 (Programming Languages) preferred
• Or permission of instructor
Lectures
• Intro I & II• Interpreters and VM’s• Objects and Types• Runtime Object
Systems• Reflection• Memory Management
• Macros• Type Inference• OO Optimizations• Partial Evaluation• Dynamic Compilation• Proof-Based
Compilation• Practical Aspects
Presentation Talking Points
• Enumerate design parameters
• Identify open problems and make progress towards their solution
• Identify and understand tradeoffs
• Note language design implications
Paper Presentations
• Each student will present one or two papers judged important to the course
• Each presentation will be about a half hour
• A reading list is available on the course web site
• Other research can be presented at instructor’s discretion
• Three slots in the following categories:
• Language Design, Interpreters & VM’s
• Types and Object Systems, Reflection, and Memory Management
• Macros, Type Inference, and OO Optimizations, Partial Evaluation and Dynamic Compilation
Panels
• Local experts
• System, Compiler, Language Design
• Positions
• Open Problems
• Secret Tricks Revealed
• Q & A
Assignments
Small one week projects– Metacircular
interpreter
– Simple VM
– Fast isa?
– Fast method dispatch
– Slot layout
in Proto such as:– Simple GC
– Type inferencer
– Specialization
– Feedback guided dispatch
– Profile guided inlining
Project
• Substantial project involving original language design and/or implementation
• Produce a design and project plan
• Working implementation
• Present an overview of their project
• 10+ page write-up
Recommended