Upload
aliza
View
42
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Programming Language Implementation Lexical and Syntax Analysis Part II. Outline. Overview of parsing Introduction Parsing Some more details Lexical analysis Parsing. Reference. - PowerPoint PPT Presentation
Citation preview
Dr. M Shafique Introduction 1
Programming Language Implementation
Lexical and Syntax Analysis
Part II
Dr. M Shafique Introduction 2
Outline
• Overview of parsing• Introduction
• Parsing
• Some more details • Lexical analysis
• Parsing
Dr. M Shafique Introduction 3
Reference
• Compilers: Principles, Techniques, and ToolsA. V. Aho, R. Sethi, and J. D. UllmanAddison-Wesley Publishing Company 1988
Chapters 1, 2, 3, 4, and 5
Dr. M Shafique Introduction 4
Introduction
• A programming language can be defined by describing its• Syntax and
• Semantics
• Grammar-oriented compilation technique • Syntax-directed translation
• Example• Infix expressions translated to post-fix expressions
• Input to output mapping
• 9 -5 +2 to 95-2+
Dr. M Shafique Introduction 5
Example
• Syntax1. e e + d
2. e e – d
3. e d
4. d 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
• Tokens• + - 0 1 2 3 4 5 6 7 8 9
Dr. M Shafique Introduction 6
Parsing
• Parsing is a process to determine if a string of tokens can be generated by a grammar
• Parsing methods• Top-down parsing
• Bottom-up parsing
Dr. M Shafique Introduction 7
Parsing• Top-down parsing
1. At node n (labeled with nonterminal A), select one of the productions for A and construct children at n for the symbols on the RHS of the production
2. Find the next node at which a subtree is to be constructed
• Recursive-descent parsing is a top-down syntax analysis method in which a set of recursive procedures are executed to process the input• A procedure is associated with each non terminal of a
grammar• Left recursive rules can loop forever
Dr. M Shafique Introduction 8
Parsing
• Bottom-up parsing
Bottom-up parsing constructs a aprse tree for an input string of tokens beginning at the leaves and working up towards the root.• Shift-reduce parsing
• Operator precedence parsing
• LR parsing
Dr. M Shafique Introduction 9
Parsing
• Removing left recursion• Example
• Left-recursive grammar
A A α | β
• Equivalent grammar without left recursion
A β R
R αR | ε
Dr. M Shafique Introduction 10
Some Important Basic Definitions
lexical: Of or relating to the morphemes of a language.
morpheme: A meaningful linguistic unit that cannotbe divided into smaller meaningful parts.
lexical analysis: The task concerned with breaking aninput into its smallest meaningful units, called tokens.
Dr. M Shafique Introduction 11
Some Important Basic Definitions
syntax: The way in which words are put togetherto form phrases, clauses, or sentences. The rulesgoverning the formation of statements in a programminglanguage.
syntax analysis: The task concerned with fitting asequence of tokens into a specified syntax.
parsing: To break a sentence down into its componentparts of speech with an explanation of the form, function,and syntactical relationship of each part.
Dr. M Shafique Introduction 12
Some Important Basic Definitions
parsing = lexical analysis + syntax analysis
semantic analysis: The task concerned with calculating the program’s meaning.
Dr. M Shafique Introduction 13
Regular Expressions
Symbol: a A regular expression formed by a.
Alternation:M | N A regular expression formed by M or N.
Concatenation:M • N A regular expression formed by M followed by N.
Epsilon: The empty string.
Repetition:M* A regular expression formed by zero or
more repetitions of M.
Dr. M Shafique Introduction 14
Building a Recognizer for a Language
General approach:
1. Build a deterministic finite automaton (DFA) from regular expression E
2. Execute the DFA to determine whether an input string belongs to L(E)
Note: The DFA construction is done automatically by a tool such as lex.
Dr. M Shafique Introduction 15
Finite Automata
A nondeterministic finite automaton A = {S, , s0, F, move }consists of:1. A set of states S2. A set of input symbols (the input symbol alphabet)3. A state s0 that is distinguished as the start state4. A state F distinguished as the accepting state5. A transition function move that maps state-symbol pairs into sets of state.
In a Deterministic Finite State Automata (DFA), the functionmove maps each state-symbol pair into a unique state.
Dr. M Shafique Introduction 16
Finite Automata
A Deterministic Finite Automaton (DFA):
A Nondeterministic Finite Automaton (NFA):
0 1 2 3a
b
b bstart
0 1 2 3
a
a
b
b bstart
What languages areaccepted by theseautomata?
b*abb
(a|b)*abb
(Aho,Sethi,Ullman, pp. 114)
Dr. M Shafique Introduction 17
Another NFA
start
a
b
a
b
An -transition is taken without consuming any character from the input.
What does the NFA above accepts?
aa*|bb*
(Aho,Sethi,Ullman, pp. 116)
Dr. M Shafique Introduction 18
Constructing NFA
It is very simple. Remember that a regular expression is formed by the use of alternation, concatenation, and repetition.
How do we define an NFA that accepts a regular expression?
Thus all we need to do is to know how to build the NFAfor a single symbol, and how to compose NFAs.
Dr. M Shafique Introduction 19
Composing NFAs with Alternation
The NFA for a symbol a is: ai fstart
Given two NFA N(s) and N(t)
N(s)
N(t)
(Aho,Sethi,Ullman, pp. 122)
starti
f
, the NFA N(s|t) is:
Dr. M Shafique Introduction 20
Composing NFAs with Concatenation
start
Given two NFA N(s) and N(t), the NFA N(st) is:
N(s) N(t)i f
(Aho,Sethi,Ullman, pp. 123)
Dr. M Shafique Introduction 21
Composing NFAs with Repetition
The NFA for N(s*) is
N(s)
fi
(Aho,Sethi,Ullman, pp. 123)
Dr. M Shafique Introduction 22
Properties of the NFA
• Following this construction rules, we obtain an NFA N(r) with these properties:• N(r) has at most twice as many states as the number of
symbols and operators in r;• N(r) has exactly one starting and one accepting state;• Each state of N(r) has at most one outgoing transition on a
symbol of the alphabet or at most two outgoing -transitions.
(Aho,Sethi,Ullman, pp. 124)
Dr. M Shafique Introduction 23
How to Parse a Regular Expression?
Given a DFA, we can generate an automaton that recognizes the longest substring of an inputthat is a valid token.
Using the three simple rules presented, it is easyto generate an NFA to recognize a regular expression.
Given a regular expression, how do we generatean automaton to recognize tokens?
Create an NFA and convert it to a DFA.
Dr. M Shafique Introduction 24
a An ordinary character stands for itself.
The empty string.
Another way to write the empty string.
M | N Alternation, Choosing from M or N.
M N Concatenation, an M followed by an N.
M* Repetition (zero or more times).
M+ Repetition (one or more times).
M? Optional, zero or one occurrence of M.
[a -zA -Z] Character set alternation.
. Stands for any single character except newline.
“a.+*” Quotation, a string in quotes stands for itself
literally.
Regular expression notation: An Example
(Appel, pp. 20)
Dr. M Shafique Introduction 25
if {return IF;}
[a - z] [a - z0 - 9 ] * {return ID;}
[0 - 9] + {return NUM;}
([0 - 9] + “.” [0 - 9] *) | ([0 - 9] * “.” [0 - 9] +) {return REAL;}
(“--” [a - z]* “\n”) | (“ ” | “ \n ” | “ \t ”) + {/* do nothing*/}
. {error ();}
(Appel, pp. 20)
Regular expressions for some tokens
Dr. M Shafique Introduction 26
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
The NFA for a symbol i is: i1 2start
The NFA for the regular expression if is:
f 31start 2i
The NFA for a symbol f is: f 2start 1
IF
if {return IF;}
Dr. M Shafique Introduction 27
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
a-z 21start
ID
[a-z] [a-z0-9 ] * {return ID;}
0-9
a-z
Dr. M Shafique Introduction 28
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
0-9 21start
NUM
[0 - 9] + {return NUM;}
0-9
Dr. M Shafique Introduction 29
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
1start
REAL
([0 - 9] + “.” [0 - 9] *) | ([0 - 9] * “.” [0 - 9] +) {return REAL;}
0-9
0-9
2 3.
0-9
0-950-94
.
Dr. M Shafique Introduction 30
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
1start
/* do nothing */
(“--” [a - z]* “\n”) | (“ ” | “ \n ” | “ \t ”) + {/* do nothing*/}
- 2
a-z
- 3 4\n
\n\t
5blank \n
\tblank
Dr. M Shafique Introduction 31
ID
1 20 - 9 0 - 9
NUM
0 - 9
1 2 3
4 5
0 - 9
0 - 9 0 - 9
0 - 9
REAL
1 2 43
5
a-z\n- -
blank, etc.blank, etc.
White space
21any but \n
error
IF
1 2a-z a-z
0-9
Building Finite Automatas for Lexical Tokens
1 2i f
3
.
.
(Appel, pp. 21)
Dr. M Shafique Introduction 32
Conversion of NFA into DFA
(Appel, pp. 27)
What states can be reached from state 1 without consuming a character?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
Dr. M Shafique Introduction 33
Conversion of NFA into DFA
What states can be reached from state 1 without consuming a character?
{1,4,9,14} form the -closure of state 1
(Appel, pp. 27)
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
Dr. M Shafique Introduction 34
Conversion of NFA into DFA
What are all the state closures in this NFA?
closure(1) = {1,4,9,14}closure(5) = {5,6,8}closure(8) = {6,8}closure(7) = {7,8}
(Appel, pp. 27)
closure(10) = {10,11,13}closure(13) = {11,13}closure(12) = {12,13}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
Dr. M Shafique Introduction 35
Conversion of NFA into DFA
Given a set of NFA states T, the -closure(T) is theset of states that are reachable through -transiton from
any state s T.
Given a set of NFA states T, move(T, a) is theset of states that are reachable on input a
from any state sT.
(Aho,Sethi,Ullman, pp. 118)
Dr. M Shafique Introduction 36
Problem Statement for Conversion of NFA into DFA
Given an NFA find the DFA with the minimum number of states that has the same behavior as the NFA for all inputs.
If the initial state in the NFA is s0, then theset of states in the DFA, Dstates, is initialized with a
state representing -closure(s0).
(Aho,Sethi,Ullman, pp. 118)
Dr. M Shafique Introduction 37
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}
1-4-9-14
Now we need to compute:
move(1-4-9-14,a-h) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
Dr. M Shafique Introduction 38
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}
1-4-9-14
Now we need to compute:
move(1-4-9-14,a-h) = {5,15}
-closure({5,15}) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
Dr. M Shafique Introduction 39
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}
1-4-9-14
Now we need to compute:
move(1-4-9-14,a-h) = {5,15}
-closure({5,15}) = {5,6,8,15}
a-h 5-6-8-15
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
Dr. M Shafique Introduction 40
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, i) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
Dr. M Shafique Introduction 41
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, i) = {2,5,15}
-closure({2,5,15}) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
Dr. M Shafique Introduction 42
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, i) = {2,5,15}
-closure({2,5,15}) = {2,5,6,8,15}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
Dr. M Shafique Introduction 43
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, j-z) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
Dr. M Shafique Introduction 44
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, j-z) = {5,15}
-closure({5,15}) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
Dr. M Shafique Introduction 45
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, j-z) = {5,15}
-closure({5,15}) = {5,6,8,15}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
j-z
Dr. M Shafique Introduction 46
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, 0-9) = {10,15}
-closure({10,15}) = {10,11,13,15}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
j-z10-11-13-15
0-9
Dr. M Shafique Introduction 47
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, other) = {15}
-closure({15}) = {15}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
j-z10-11-13-15
0-9
15other
Dr. M Shafique Introduction 48
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}
The analysis for 1-4-9-14is complete. We mark it andpick another state in the DFAto analyse.
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
j-z10-11-13-15
0-9
15other
Dr. M Shafique Introduction 49
The corresponding DFA
5-6-8-15
2-5-6-8-15
10-11-13-15
3-6-7-8
11-12-13
6-7-8
15
1-4-9-14
a-e, g-z, 0-9
a-z,0-9
a-z,0-9
0-9
0-9
f
i
a-h
j-z
0-9
other
ID
ID
NUM NUM
IF
error
ID
a-z,0-9
(Appel, pp. 29)
See pp. 118 of Aho-Sethi-Ullmanand pp. 29 of Appel.
Dr. M Shafique Introduction 50
Lexical Analyzer and Parser
lexicalanalyzer
Syntaxanalyzer
symboltable
get nexttoken
(Aho,Sethi,Ullman, pp. 160)
token: smallest meaningful sequence of characters of interest in source program
SourceProgram
get nextchar
next char next token
(Contains a record for each identifier)
Dr. M Shafique Introduction 51
Definition of Context-Free Grammars
A context-free grammar G = (T, N, S, P) consists of:1. T, a set of terminals (scanner tokens).2. N, a set of nonterminals (syntactic variables generated
by productions).3. S, a designated start nonterminal.4. P, a set of productions. Each production has the form,
A::= , where A is a nonterminal and is a sentential form , i.e., a string of zero or more grammar symbols (terminals/nonterminals).
Dr. M Shafique Introduction 52
Syntax Analysis
Syntax Analysis Problem Statement: To find a derivation sequence in a grammar G for the input token stream (or say that none exists).
Dr. M Shafique Introduction 53
Tree nodes represent symbols of the grammar (nonterminals or terminals) and tree edges represent derivation steps.
Parse trees
A parse tree is a graphical representation of a derivation sequence of a sentential form.
Dr. M Shafique Introduction 54
Derivation
E E + E | E E | ( E ) | - E | id
Given the following grammar:
Is the string -(id + id) a sentence in this grammar?
Yes because there is the following derivation:
E -E -(E) -(E + E) -(id + id)
Where reads “derives in one step”.
(Aho,Sethi,Ullman, pp. 168)
Dr. M Shafique Introduction 55
Derivation
E E + E | E E | ( E ) | - E | id
Lets examine this derivation:
E -E -(E) -(E + E) -(id + id)
E E
E-
E
E-
E( )
E
E-
E( )
+E E
E
E-
E( )
+E E
id idThis is a top-down derivationbecause we start building theparse tree at the top parse tree
(Aho,Sethi,Ullman, pp. 170)
Dr. M Shafique Introduction 56Which derivation tree is correct?
Another Derivation Example
Find a derivation for the expression: id + id idE E
+E E
E
+E E
E E
E
+E E
E E
id id
id
E E
E E
E
E E
+E E
E
E E
+E E
id id
id
E E + E | E E | ( E ) | - E | id
(Aho,Sethi,Ullman, pp. 171)
Dr. M Shafique Introduction 57
According to the grammar, both are correct.
Another Derivation Example
Find a derivation for the expression: id + id idE
+E E
E E
id id
id
E
+E E
E E
id id
id
A grammar that produces more than oneparse tree for any input sentence is saidto be an ambiguous grammar.
E E + E | E E | ( E ) | - E | id
(Aho,Sethi,Ullman, pp. 171)
Dr. M Shafique Introduction 58
Left Recursion
Consider the grammar:E E + T | TT T F | FF ( E ) | id
A top-down parser might loop forever when parsingan expression using this grammar
E E
+E T
E
+E T
+E T
E
+E T
+E T
+E T
(Aho,Sethi,Ullman, pp. 176)
Dr. M Shafique Introduction 59
Left Recursion
Consider the grammar:E E + T | TT T F | FF ( E ) | id
A grammar that has at least one production of the formA A is a left recursive grammar.
Top-down parsers do not work with left-recursivegrammars.
Left-recursion can often be eliminated by rewriting thegrammar.
(Aho,Sethi,Ullman, pp. 176)
Dr. M Shafique Introduction 60
Left Recursion
This left-recursivegrammar:
E E + T | TT T F | FF ( E ) | id
Can be re-written to eliminate the immediate left recursion:
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
(Aho,Sethi,Ullman, pp. 176)
Dr. M Shafique Introduction 61
Predictive Parsing
Consider the grammar:
stm if expr then stmt else stmt | while expr do stmt | begin stmt_list end
A parser for this grammar can be written with the following simple structure: switch(gettoken())
{ case if: …. break;
case while: …. break;
case begin: …. break;
default: reject input;}
Based only on the first token,the parser knows which rule to use to derive a statement.
Therefore this is called apredictive parser.
(Aho,Sethi,Ullman, pp. 183)
Dr. M Shafique Introduction 62
Left Factoring
The following grammar:
stmt if expr then stmt else stmt | if expr then stmt
Cannot be parsed by a predictive parser that looksone element ahead.
But the grammar can be re-written:
stmt if expr then stmt stmt’stmt‘ else stmt |
Where is the empty string.
(Aho,Sethi,Ullman, pp. 178)
Rewriting a grammar to eliminate multiple productionsstarting with the same token is called left factoring.
Dr. M Shafique Introduction 63
A Predictive Parser
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
Grammar:
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
ParsingTable:
(Aho,Sethi,Ullman, pp. 188)
Dr. M Shafique Introduction 64
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
E
$
$ OUTPUT:
E
T
E’
$
T E’
PARSINGTABLE:
Dr. M Shafique Introduction 65
T
E’
$
T
E’
$
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
$ OUTPUT:
E
F
T’
E’
$
F T’
T E’
PARSINGTABLE: (Aho,Sethi,
Ullman, pp. 186)
Dr. M Shafique Introduction 66
(Aho,Sethi,Ullman, pp. 188)
T
E’
$
T
E’
$
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
$ OUTPUT:
E
F
T’
E’
$
F T’
T E’
id
T’
E’
$id
PARSINGTABLE:
Dr. M Shafique Introduction 67
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
$ OUTPUT:
E
F
T’
E’
$
F T’
T E’
id
T’
E’
$id
Action when Top(Stack) = input $ : Pop stack, advance input.
PARSINGTABLE: (Aho,Sethi,
Ullman, pp. 188)
Dr. M Shafique Introduction 68
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
$ OUTPUT:
E
F T’
T E’
id
T’
E’
$
E’
$
PARSINGTABLE: (Aho,Sethi,
Ullman, pp. 188)
Dr. M Shafique Introduction 69
A Predictive Parser
E
F T’
T E’
id
T+ E’
F T’
id F T’
id
The predictive parser proceedsin this fashion emiting thefollowing productions:
E’ +TE’T FT’F idT’ FT’F idT’ E’
When Top(Stack) = input = $the parser halts and accepts the
input string. (Aho,Sethi,Ullman, pp. 188)
Dr. M Shafique Introduction 70
LL(k) Parser
This parser parses from left to right, and does aleftmost-derivation. It looks up 1 symbol ahead to choose its next action. Therefore, it is known asa LL(1) parser.
An LL(k) parser looks k symbols ahead to decideits action.
Dr. M Shafique Introduction 71
The Parsing Table
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
Given this grammar:
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
PARSINGTABLE:
How is this parsing table built?
Dr. M Shafique Introduction 72
FIRST and FOLLOW
We need to build a FIRST set and a FOLLOW setfor each symbol in the grammar.
FIRST() is the set of terminal symbols that can begin any string derived from .
The elements of FIRST and FOLLOW areterminal symbols.
FOLLOW() is the set of terminal symbols that can follow :
t FOLLOW() derivation containing t
(Aho,Sethi,Ullman, pp. 189)
Dr. M Shafique Introduction 73
Rules to Create FIRST
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If X is a terminal, FIRST(X) = {X}
FIRST(id) = {id}FIRST() = {}FIRST(+) = {+}
SETS:
2. If X , then FIRST(X)3. If X Y1Y2 ••• Yk
FIRST(() = {(}FIRST()) = {)}
FIRST rules:
*and Y1 ••• Yi-1 and a FIRST(Yi)
then a FIRST(X)
FIRST(F) = {(, id}FIRST(T) = FIRST(F) = {(, id}FIRST(E) = FIRST(T) = {(, id}
FIRST(E’) = {} {+, }FIRST(T’) = {} {, }
(Aho,Sethi,Ullman, pp. 189)
Dr. M Shafique Introduction 74
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {$}
FOLLOW(E’) = { ), $}
SETS:
2. If A B, and a FIRST() and a then a FOLLOW(B)3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW rules:
{ ), $}
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
* FOLLOW(T) = { ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
A and B are non-terminals, and are strings of grammar symbols
(Aho,Sethi,Ullman, pp. 189)
Dr. M Shafique Introduction 75
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
SETS: 3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW rules:
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
* FOLLOW(T) = { ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
2. If A B, and a FIRST() and a then a FOLLOW(B)
{+, ), $}
(Aho,Sethi,Ullman, pp. 189)
Dr. M Shafique Introduction 76
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
SETS:
FOLLOW rules:
FOLLOW(T) = {+, ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
2. If A B, and a FIRST() and a then a FOLLOW(B)3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW(T’) = {+, ), $}
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
*
(Aho,Sethi,Ullman, pp. 189)
Dr. M Shafique Introduction 77
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
SETS:
FOLLOW rules:
FOLLOW(T) = {+, ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
2. If A B, and a FIRST() and a then a FOLLOW(B)3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW(T’) = {+, ), $}
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
* FOLLOW(F) = {+, ), $}
(Aho,Sethi,Ullman, pp. 189)
Dr. M Shafique Introduction 78
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
SETS:
FOLLOW rules:
FOLLOW(T) = {+, ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW(T’) = {+, ), $}
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
* FOLLOW(F) = {+, ), $}
2. If A B, and a FIRST() and a then a FOLLOW(B)
{+, , ), $}
(Aho,Sethi,Ullman, pp. 189)
Dr. M Shafique Introduction 79
Rules to Build Parsing Table
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Dr. M Shafique Introduction 80
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
Dr. M Shafique Introduction 81
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
Dr. M Shafique Introduction 82
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
Dr. M Shafique Introduction 83
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
Dr. M Shafique Introduction 84
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]2. If A : if FIRST(), add A to M[A, b] for each terminal b FOLLOW(A),
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
Dr. M Shafique Introduction 85
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]2. If A : if FIRST(), add A to M[A, b] for each terminal b FOLLOW(A),
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
Dr. M Shafique Introduction 86
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]2. If A : if FIRST(), add A to M[A, b] for each terminal b FOLLOW(A), 3. If A : if FIRST(), and $ FOLLOW(A), add A to M[A, $]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
Dr. M Shafique Introduction 87
Bottom-Up and Top-Down Parsers
Top-down parsers: starts constructing the parse tree at thetop (root) of the tree and move down towards the leaves.• Easy to implement by hand, but • Works with restricted grammars.
example: predictive parsers
Bottom-up parsers: builds the nodes on the bottom of theparse tree first and moves up towards the root.• Suitable for automatic parser generation• Handle a larger class of grammars.
examples: shift-reduce parser (or LR(k) parsers)
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 88
Bottom-Up Parser
A bottom-up parser, or a shift-reduce parser, beginsat the leaves and works up to the top of the tree.
The reduction steps trace a rightmost derivationin reverse.
S aABeA Abc | bB d
Consider the Grammar:
We want to parse the input string abbcde.
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 89
Bottom-Up Parser Example
a dbb cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:$
ProductionS aABeA Abc
A bB d
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 90
Bottom-Up Parser Example
a dbb cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A
b
$
ProductionS aABeA Abc
A bB d
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 91
Bottom-Up Parser Example
a dbA cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A
b
$
ProductionS aABeA Abc
A bB d
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 92
Bottom-Up Parser Example
a dbA cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A
b
$
ProductionS aABeA Abc
A bB d
We are not reducing here in this example.
A parser would reduce, get stuck and then backtrack!
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 93
Bottom-Up Parser Example
a dbA cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A
b
$
ProductionS aABeA Abc
A bB d
c
A
b
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 94
Bottom-Up Parser Example
a dAINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 95
Bottom-Up Parser Example
a dAINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
B
d
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 96
Bottom-Up Parser Example
a BAINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
B
d
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 97
Bottom-Up Parser Example
a BAINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
B
d
a
S
e
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 98
Bottom-Up Parser Example
SINPUT:
Bottom-Up ParsingProgram
OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
B
d
a
S
e
This parser is known as an LR Parser because it scans the input from Left to right, and it constructs
a Rightmost derivation in reverse order.
(Aho,Sethi,Ullman, pp. 195)
Dr. M Shafique Introduction 99
Bottom-Up Parser Example
The scanning of productions for matching withhandles in the input string, and backtracking makesthe method used in the previous example veryinefficient.
Can we do better?
Dr. M Shafique Introduction 100
LR Parser Example
Input
Stack
LR ParsingProgram
action goto
Output
(Aho,Sethi,Ullman, pp. 217)
Dr. M Shafique Introduction 101
LR Parser Example
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
The following grammar:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
Can be parsed with this actionand goto table
(Aho,Sethi,Ullman, pp. 219)
Dr. M Shafique Introduction 102
LR Parser Example
id idid +INPUT: $
STACK: E0
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
GRAMMAR:
OUTPUT:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 103
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E5
id
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 104
OUTPUT:
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 105
OUTPUT:
E3
F
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 106
OUTPUT:
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 107
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E2
T
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 108
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
E7
2
T
0
T
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 109
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E5
id
7
2
T
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 110
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E7
2
T
0action goto State
id + * ( ) $ E T F 0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 111
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E10
F
7
2
T
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 112
OUTPUT:
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 113
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
2
T
0
T
T F
F
id
idaction goto State
id + * ( ) $ E T F 0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
E
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 114
OUTPUT:
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
E
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 115
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
1
E
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
E
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 116
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
E
6
+
1
E
0
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 117
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
5
id
6
+
1
E
0
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 118
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
OUTPUT:
T
T F
F
id
id
E
6
+
1
E
0
F
id
GRAMMAR:
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 119
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
3
F
6
+
1
E
0
F
id
GRAMMAR:
T
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 120
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
OUTPUT:
T
T F
F
id
id
E
6
+
1
E
0
F
id
GRAMMAR:
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 121
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
9
T
6
+
1
E
0
F
id
GRAMMAR:
T
E
+
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 122
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
0
GRAMMAR:
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
F
id
T
E
+
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 123
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
1
E
0
F
id
GRAMMAR:
T
E
+
(Aho,Sethi,Ullman, pp. 220)
Dr. M Shafique Introduction 124
Constructing Parsing Tables
All LR parsers use the same parsing program thatwe demonstrated in the previous slides.
What differentiates the LR parsers are the action and the goto tables:
Simple LR (SLR): succeeds for the fewest grammars, but is the easiest to implement.
Canonical LR: succeeds for the most grammars, but is the hardest to implement. It splits states when necessary to prevent reductions that would get the parser stuck.
Lookahead LR (LALR): succeeds for most common syntacticconstructions used in programming languages, but producesLR tables much smaller than canonical LR.
(See AhoSethiUllman pp. 221-230).
(See AhoSethiUllman pp. 236-247).
(See AhoSethiUllman pp. 230-236).
(Aho,Sethi,Ullman, pp. 221)
Dr. M Shafique Introduction 125
Using Lex
Lexcompiler
Lexsource
programlex.l
lex.yy.c
Ccompiler
lex.yy.c a.out
a.outInput
stream
sequenceof
tokens
(Aho-Sethi-Ullman, pp. 258)
Dr. M Shafique Introduction 126
Using Yacc
Yacccompiler
YaccSpecificationtranslate.y
y.tab.c
Ccompiler
y.tab.c a.out
a.outInput
output
(Aho-Sethi-Ullman, pp. 258)
yacc translate.y
cc y.tab.c -ly
Dr. M Shafique Introduction 127
Parsing Action Conflicts
If the grammar specified is ambiguous, yacc willreport parsing action conflicts.
These conflicts can be reduce/reduce conflicts orshift/reduce conflicts.
Yacc has rules to resolve such conflicts automatically(see AhoSethiUllman, pp. 262-264), but the resultingparser might not have the behavior intended by thegrammar writer.
Whenever you see a conflict report, rerun yacc withthe -v flag, examine the y.output file, and re-writeyour grammar to eliminate the conflicts.
(Aho-Sethi-Ullman, pp. 262)
Dr. M Shafique Introduction 128
Three-Address StatementsA popular form of intermediate code used in optimizing compilers is three-address statements (or variations, such as quadruples).
Source statement:x = a + b c + d
Three address statements with temporaries t1 and t2:
t1 = b ct2 = a + t1
x = t2 + d
• Intermediate Code GenerationRead Aho-Sethi-Ullman: Chapter 8.1 ~ 8.3, 8.7
(Aho-Sethi-Ullman, pp. 466)
Dr. M Shafique Introduction 129
Lexical Analyzer (Scanner)+
Syntax Analyzer (Parser)+ Semantic Analyzer
Abstract Syntax Tree with attributes
Intermediate-code Generator
Non-optimized Intermediate Code
FrontEnd
ErrorMessage
Front End of a Compiler
Dr. M Shafique Introduction 130
Component-Based Approach to Building Compilers
Target-1 Code Generator Target-2 Code Generator
Intermediate-code Optimizer
Language-1 Front End
Source programin Language-1
Language-2 Front End
Source programin Language-2
Non-optimized Intermediate Code
Optimized Intermediate Code
Target-1 machine code Target-2 machine code
Dr. M Shafique Introduction 131
Advantages of Using an Intermediate Language
1. Retargeting - Build a compiler for a new machine by attaching a new code generator to an existing front-end.
2. Optimization - reuse intermediate code optimizers in compilers for different languages and different machines.
Note: the terms “intermediate code”, “intermediate language”, and “intermediate representation” are all used interchangeably.
Dr. M Shafique Introduction 132
position := initial + rate * 60
Th
e P
has
es o
f a
Co
mp
iler
lexical analyzer
id1 := id2 + id3 * 60
syntax analyzer
:=
id1 +
id2 *
id3 60
semantic analyzer
:=
id1 +
id2 *
id3 inttoreal
60
intermediate code generator
temp1 := inttoreal (60)temp2 := id3 * temp1temp3 := id2 + temp2id1 := temp3
code optimizer
temp1 := id3 * 60.0id1 := id2 + temp1
code generator
MOVF id3, R2MULF #60.0, R2MOVF id2, R1ADDF R2, R1MOVF R1, id1