Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Compiler and Language Processing ToolsSummer Term 2011
Introduction
Prof. Dr. Arnd Poetzsch-Heffter
Software Technology GroupTU Kaiserslautern
Prof. Dr. Arnd Poetzsch-Heffter Compilers 1
Introduction
Outline
1. IntroductionLanguage Processing ToolsApplication DomainsTasks of Language-Processing ToolsExamples
2. Language ProcessingTerminology and RequirementsCompiler Architecture
3. Compiler Construction
Prof. Dr. Arnd Poetzsch-Heffter Compilers 2
Introduction Language Processing Tools
Language processing tools
• Processing of source texts in (source) languages
• Analysis of (source) texts
• Translation to target languages
Prof. Dr. Arnd Poetzsch-Heffter Compilers 3
Introduction Language Processing Tools
Language processing tools (2)
Typical source languages• Programming languages: C, C++, C#, Java, Scala, Haskell, ML,
Smalltalk, Prolog• Script languages: JavaScript, bash• Languages for configuration management: make, ant• Application and tool-specific languages: Excel, JFlex, CUPS• Specification languages: Z, CASL, Isabelle/HOL• Formatting and data description languages: LaTeX, HTML, XML• Design and architecture description languages: UML, SDL, VHDL,
Verilog
Prof. Dr. Arnd Poetzsch-Heffter Compilers 4
Introduction Language Processing Tools
Language processing tools (3)
Typical target languages
• Assembly, machine, and bytecode languages
• Programming language
• Data and layout description languages
• Languages for printer control
• ...
Prof. Dr. Arnd Poetzsch-Heffter Compilers 5
Introduction Language Processing Tools
Language processing tools (4)
Language implementation tasks
• Tool support for language processing
• Integration into existing systems
• Connection to other systems
Prof. Dr. Arnd Poetzsch-Heffter Compilers 6
Introduction Application Domains
Application domains
• Programming environmentsI Context-sensitive editors, class browersI Graphical programming toolsI Pre-processorsI CompilersI InterpretersI DebuggersI Run-time environments (loading, linking, execution, memory
management)
Prof. Dr. Arnd Poetzsch-Heffter Compilers 7
Introduction Application Domains
Application domains (2)
• Generation of programs from design documents (UML)• Program comprehension, re-engineering• Design and implementation of domain-specific languages
I Robot controlI Simulation toolsI Spread sheets, active documents
• Web technologyI Analysis of Web sitesI Active Web sites (with integrated functionality)I Abstract platforms, e.g. JVM, .NETI Optimization of caching
Prof. Dr. Arnd Poetzsch-Heffter Compilers 8
Introduction Application Domains
Related fields
• Formal languages, language specification and design• Programming and specification languages• Programming, software engineering, software generation,
software architecture• System software, computer architecture
Prof. Dr. Arnd Poetzsch-Heffter Compilers 9
Introduction Tasks of Language-Processing Tools
Tasks of Language-Processing Tools
Analyser Translation Interpreter
Source Code Source Code
Target CodeAnalysis Results
Source Code
Input Data
Output Data
Analysis, translation and interpretation are often combined.
Prof. Dr. Arnd Poetzsch-Heffter Compilers 10
Introduction Tasks of Language-Processing Tools
Tasks of Language-Processing Tools (2)
1. TranslationI Compiler implements analysis and translationI OS and real machine implement interpretation
Pros:I Most efficient solutionI One interpreter for different programming languagesI Prerequisite for other solutions
Prof. Dr. Arnd Poetzsch-Heffter Compilers 11
Introduction Tasks of Language-Processing Tools
Tasks of Language-Processing Tools (3)
2. Direct interpretationI Interpreter implements all tasks.I Examples: JavaScript, command line languages (bash)I Pros: No translation necessary (but analysis at run-time)
Prof. Dr. Arnd Poetzsch-Heffter Compilers 12
Introduction Tasks of Language-Processing Tools
Tasks of Language-Processing Tools (4)
3. Abstract and virtual machinesI Compiler implements analysis and translation to abstract machine
codeI Abstract machine works as interpreterI Examples: Java/JVM, C#, .NETI Pros:
• Platform independent (portability, mobile code)• Self-modifing programs possible
4. Other combinations
Prof. Dr. Arnd Poetzsch-Heffter Compilers 13
Introduction Examples
Example: Analysis
17.04.2007 8© A. Poetzsch-Heffter, TU Kaiserslautern
package b1_1
;
class Weltklasse
extends Superklasse
implement BesteBohnen
{Qualifikation studieren
( Arbeit schweiss) {
return new
Qualifikation
();}}
Beispiel: (Analyse)
javac-Analysator
Superklasse.class
Qualifikation.class
Arbeit.class
BesteBohnen.class
...
b1_1/Weltklasse.java:4: '{' expected.
extends Superklasse
^
1 error
Prof. Dr. Arnd Poetzsch-Heffter Compilers 14
Introduction Examples
Example: Translation
17.04.2007 9© A. Poetzsch-Heffter, TU Kaiserslautern
package b1_1;
class Weltklasse
extends Superklasse
implements BesteBohnen
{
Qualifikation studieren
( Arbeit schweiss ) {
return new Qualifikation();
}}
Beispiel 1: (Übersetzung)
javac
Su
pe
rkla
sse.
cla
ss
Qu
alif
ika
tion.
cla
ss
Arb
eit.
clas
s
Be
ste
Bo
hne
n.c
lass
...
Compiled from Weltklasse.java
class b1_1/Weltklasse
extends ... implements ... {
b1_1/Weltklasse();
b1_1.Qualifikation studieren(...);
}
Method b1_1/Weltklasse()
...
Method b1_1.Qualifikation studieren(...)
...
Prof. Dr. Arnd Poetzsch-Heffter Compilers 15
Introduction Examples
Example: Translation (2)Result of translation
17.04.2007
10
© A
. Poetzsch-H
effter, TU
Kaiserslautern
Be
isp
iel 1
:(F
orts
etz
un
g)
Compiled from Weltklasse.java
class b1_1/Weltklasse
extends b1_1.Superklasse
implements b1_1.BesteBohnen {
b1_1/Weltklasse();
b1_1.Qualifikation studieren(b1_1.Arbeit);
}
Method b1_1/Weltklasse()
0 aload_0
1 invokespecial #6 <Method b1_1.Superklasse()>
4 return
Method b1_1.Qualifikation studieren(b1_1.Arbeit)
0 new #2 <Class b1_1.Qualifikation>
3 dup
4 invokespecial #5 <Method b1_1.Qualifikation()>
7 areturn
Prof. Dr. Arnd Poetzsch-Heffter Compilers 16
Introduction Examples
Example 2: Translation
17.04.2007 11© A. Poetzsch-Heffter, TU Kaiserslautern
int main() {
printf("Willkommen zur Vorlesung!");
return 0;
}
Beispiel 2: (Übersetzung)
gcc
.file "hello_world.c"
.version "01.01"
gcc2_compiled.:
.section .rodata
.LC0:
.string "Willkommen zur Vorlesung!"
.text
.align 16
.globl main
.type main,@function
main:
pushl %ebp
movl %esp,%ebp
subl $8,%esp
...
Prof. Dr. Arnd Poetzsch-Heffter Compilers 17
Introduction Examples
Example 2: Translation (2)Result of translation
17.04.2007 12© A. Poetzsch-Heffter, TU Kaiserslautern
Beispiel 2: (Fortsetzung)
.file "hello_world.c"
.version "01.01"
gcc2_compiled.:
.section .rodata
.LC0:
.string "Willkommen zur Vorlesung!"
.text
.align 16
.globl main
.type main,@function
main:
pushl %ebp
movl %esp,%ebp
subl $8,%esp
addl $-12,%esp
pushl $.LC0
call printf
addl $16,%esp
xorl %eax,%eax
jmp .L2
.p2align 4,,7
.L2:
movl %ebp,%esp
popl %ebp
ret
.Lfe1:
.size main,.Lfe1-main
.ident "GCC: (GNU) 2.95.2 19991024 (release)"
Prof. Dr. Arnd Poetzsch-Heffter Compilers 18
Introduction Examples
Example 3: Translation
17.04.2007 13© A. Poetzsch-Heffter, TU Kaiserslautern
Beispiel 3: (Übersetzung)
\documentclass{article}
\begin{document}
\vspace*{7cm}
\centerline{\Huge\bf It‘s groovy}
\end{document}
groovy.tex (104 bytes)
...
groovy.dvi (207 bytes, binary)
%!PS-Adobe-2.0
%%Creator: dvips(k) 5.86 ...
%%Title: groovy.dvi
...
groovy.ps (7136 bytes)
latex
dvips
Prof. Dr. Arnd Poetzsch-Heffter Compilers 19
Introduction Examples
Example: Interpretation
17.04.2007 14© A. Poetzsch-Heffter, TU Kaiserslautern
Beispiel: (Interpretation)
...
14 iload_1
15 iload_2
16 idiv
17 istore_3
...
.class-Datei
Eingabedaten
Ausgabedaten
...
14 iload_1
15 iload_2
16 idiv
17 istore_3
...
Java Virtual Machine (JVM)
Input Data
Output Data
.class File
Prof. Dr. Arnd Poetzsch-Heffter Compilers 20
Introduction Examples
Example: Combined technique
Java implementation with just-in-time (JIT) compiler
17.04.2007 15© A. Poetzsch-Heffter, TU Kaiserslautern
Kombinierte Implementierungstechnik: Java-Implementierung mit JIT-Übersetzer
Java-Überset-zungseinheit
java
c Analysator
Übersetzer
Eingabedaten
Java Byte Code
.class-Datei
Ausgabedaten
JIT-Übersetzer
JVM
Maschinencode reale Maschine/Hardware
(JIT=Just in time)
Beispiel: (Kombinierte Technik)
Java SourceCode Unit
AnalyzerTranslator
Input Data
Output Data
.class file
JIT Translator
Machine Code Real Machine / Hardware
Prof. Dr. Arnd Poetzsch-Heffter Compilers 21
Language Processing Terminology and Requirements
Language processing: The task of translation
Translator
Source Code
Error Message or Target Code
• Translator (in a broader sense):Analysis, optimization andtranslation
• Source code:Input (string) for translator insyntax of source language (SL)
• Target Code:Output (string) of translator insyntax of target language (TL)
Prof. Dr. Arnd Poetzsch-Heffter Compilers 22
Language Processing Terminology and Requirements
Phases of language processing
• Analysis of input:I Program textI SpecificationI Diagrams
• Dependant on target of implementationI Transformation (XSLT, refactoring)I Pretty printing, formattingI Semantic analysis (program comprehension)I OptimizationI (Actual) translation
Prof. Dr. Arnd Poetzsch-Heffter Compilers 23
Language Processing Terminology and Requirements
Compile time vs. run-time
• Compile time: during run-time of compiler/translatorStatic: All information/aspects known at compile time, e.g.:
I Type checksI Evaluation of constant expressionsI Relative addresses
• Run-time: during run-time of compiled programDynamic: All information that are not statically known, e.g.:
I Allocation of dynamic arraysI Bounds check of arraysI Dynamic binding of methodsI Memory management of recursive procedures
For dynamic aspects that cannot be handled at compile time, thecompiler generates code that handles these aspects at run-time.
Prof. Dr. Arnd Poetzsch-Heffter Compilers 24
Language Processing Terminology and Requirements
What is a good compiler?
Prof. Dr. Arnd Poetzsch-Heffter Compilers 25
Language Processing Terminology and Requirements
Requirements for translators
• Error handling (static/dynamic)• Efficient target code• Choice: Fast translation with slow code
vs. slow translation with fast code• Semantically correct translation
Prof. Dr. Arnd Poetzsch-Heffter Compilers 26
Language Processing Terminology and Requirements
Semantically correct translation
Intuitive definition: Compiled program behaves according tolanguage definition of source language.
Formal definition:• semSL: SL_Program × SL_Data→ SL_Data• semTL: TL_Program × TL_Data→ TL_Data• compile: SL_Program→ TL_Program• code: SL_Data→ TL_Data• decode: TL_Data→ SL_Data
Semantic correctness:semSL(P,D) = decode(semTL(compile(P), code(D)))
Prof. Dr. Arnd Poetzsch-Heffter Compilers 27
Language Processing Compiler Architecture
Compiler Architecture
Scanner
Source Codeas String
TokenStream
Parser
Name and Type Analysis
Translator
Code Generator
Syntax Tree
DecoratedSyntax Tree
(Close to SL)
IntermediateLanguage
Target Codeas String
Attribution &Optimization
Attribution &Optimization
Peep HoleOptimization
Analysis
Synthesis
Prof. Dr. Arnd Poetzsch-Heffter Compilers 28
Language Processing Compiler Architecture
Properties of compiler architectures
• Phases are conceptual units of translation• Phases can be interleaved• Design of phases depends on source language, target language
and design decisions• Phase vs. pass (phase can comprise more than one pass.)• Separate translation of pogram parts
(Interface information must be accessible.)• Combination with other architecture decisions:
Common intermediate language
Prof. Dr. Arnd Poetzsch-Heffter Compilers 29
Language Processing Compiler Architecture
Common intermediate language
Source Language 1
Source Language 2
Source Language n
Intermediate Language
Target Language 1
Target Language 2
Target Language m
...
...
Prof. Dr. Arnd Poetzsch-Heffter Compilers 30
Language Processing Compiler Architecture
Dimensions of compiler construction
• Programming languagesI Sequential procedural, imperative, OO-languagesI Functional, logical languagesI Parallel languages/language constructs
• Target languages/machinesI Code for abstract machinesI AssemblerI Machine languages (CISC, RISC, ...)I Multi-processor/multi-core architecturesI Memory hierarchy
• Translation tasks: analysis, optimization, synthesis• Construction techniques and tools: bootstrapping, generators• Portability, specification, correctness
Prof. Dr. Arnd Poetzsch-Heffter Compilers 31
Compiler Construction
Compiler construction techniques
1. Stepwise constructionI Construction with compiler for different languageI Construction with compiler for different machineI Bootstrapping
2. Compiler-compiler: Tools for compiler generationI Scanner generators (regular expressions)I Parser generators (context-free grammars)I Attribute evaluation generators (attribute grammar)I Code generator generators (machine specification)I Interpreter generators (semantics of language)I Other phase-specific tools
3. Special programming techniquesI General technique: syntax-drivenI Special technique: recursive descend
Prof. Dr. Arnd Poetzsch-Heffter Compilers 32
Compiler Construction
Stepwise construction
Programming typically depends on an existing compiler for theimplementation language. For compiler construction, this does not holdin general.
Source, target, and implementation languages of compilers can bedenoted in T-diagrams.
CL
SL TL
T-diagram denotes compiler from source language SL to targetlanguage TL (SL→ TL compiler) written in language CL.
Prof. Dr. Arnd Poetzsch-Heffter Compilers 33
Compiler Construction
Construction with compiler for different language
• Given: C → ML (machine language) compiler in ML• Construct: SL→ ML compiler in ML• Solution: Develop SL→ ML compiler in C, translate that compiler
from C → ML by using the existing C → ML compiler
C
SL ML
ML
C ML ML
SL ML
to be developed
existing
by translation
Prof. Dr. Arnd Poetzsch-Heffter Compilers 34
Compiler Construction
Construction with compiler for different machine
• Construct: C → ML1 compiler in ML1• Given
1. C → ML1 compiler in C2. C → ML2 compiler in ML2
• Method: construct cross compiler
First step
C
C ML1
ML2
C ML2 ML2
C ML1
cross compilergiven
given
Prof. Dr. Arnd Poetzsch-Heffter Compilers 35
Compiler Construction
Construction with compiler for different machine (2)
Second step
C
C ML1
ML2
C ML1 ML1
C ML1
resulting compilergiven
cross compiler
Prof. Dr. Arnd Poetzsch-Heffter Compilers 36
Compiler Construction
Bootstrapping
• Construct: SL→ ML compiler in ML• Suppose: yet no compiler exists• Method:
1. Construct partial language SLi of SL such thatSL0 ⊂ SL1 ⊂ SL2 ⊂ . . . ⊂ SL
2. Implement SL0 compiler for ML in ML3. Implement SLi+1 compiler for ML in SLi
4. Create SLi+1 compiler for ML in ML
Prof. Dr. Arnd Poetzsch-Heffter Compilers 37
Compiler Construction
Bootstrapping (2)
SL0
SL1 ML
ML
SL0 ML ML
SL1 MLSL1
SL2 ML SL2
SL ML
ML
SL2 ML ML
SL ML
manually
by extension
by translation
Prof. Dr. Arnd Poetzsch-Heffter Compilers 38
Compiler Construction
Recommended reading
Wilhelm, Maurer:• Chap. 1, Introduction (pp. 1–5)• Chap. 6, Structure of Compilers (pp. 225 – 238)
Appel• Chap. 1, Introduction (pp. 3 – 14)
Prof. Dr. Arnd Poetzsch-Heffter Compilers 39