Transcript ppt

Announcements

HW1 due today

HW2 will be out tonight


Due 2/18
LL and LR grammars and parsing
Spring 16 CSCI 4430, A Milanova
1
Last Class

Top-down (LL) parsing



LL(1) parsing tables, FIRST, FOLLOW and
PREDICT sets
Writing an LL(1) grammar
Bottom-up (LR) parsing

Model the LR parser
Spring 16 CSCI 4430, A Milanova
2
Today’s Lecture Outline

Bottom-up (LR) parsing

Model of the LR parser

LR Items
Characteristic Finite State Machine (CFSM)
SLR(1) parsing table
LR parsing variants



Spring 16 CSCI 4430, A Milanova
3
Programming Language Syntax
Bottom-up Parsing
Read: Scott, Chapter 2.3.3
4
id + id*id
Stack
Input
id+id*id
id
+id*id
term
+id*id
expr
+id*id
expr+
id*id
expr+id
*id
Spring 16 CSCI 4430, A Milanova
expr  expr + term | term
term  term * id | id
Action
shift id
reduce by term id
reduce by expr term
shift +
shift id
reduce by term  id
5
expr  expr + term | term
term  term * id | id
id + id*id
Stack
Input Action
expr+term
*id
expr+term*
id
expr+term*id
expr+term
expr
Spring 16 CSCI 4430, A Milanova
shift *
shift id
reduce by termterm*id
reduce by exprexpr+term
accept, SUCCESS
6
id + id*id
expr  expr + term | term
term  term * id | id
Sequence of reductions performed by parser
id+id*id
• A rightmost derivation in
reverse
term+id*id
expr+id*id
• The stack (e.g., expr)
concatenated with remaining
expr+term*id
input (e.g., +id*id) gives a
expr+term
expr
Spring 16 CSCI 4430, A Milanova
sentential form (expr+id*id)
in the rightmost derivation.
• I call valid sentential forms
in rightmost derivations right
sentential forms.
7
Notation:
A,S are nonterminals.
α,β are arbitrary sequences
of terminals and nonterminals.
w is a string of terminals.
Handle

A handle


If we have a rightmost derivation
S …  αAw  αβw, then we say that
A  β at position α is a handle of αβw
Recall our example id+id*id
Stack
expr+term
expr+term*id
Spring 16 CSCI 4430, A Milanova
Input
*id
Is expr  expr+term a handle
of expr+term*id at
position ε?
Is term  id a handle of
expr+term*id at position
expr+term* ?
8
Question

expr  expr + term | term
term  term * id | id
Consider id*id*id
Stack
term

Input
*id*id
Is expr  term a
handle of term*id*id
at position ε?
Answer: No! It brings sentential form
term*id*id into expr*id*id which is not
derivable in a rightmost derivation (You cannot
derive sentential form expr*id*id from expr!)
Spring 16 CSCI 4430, A Milanova
9
Question
How about
Stack
Input
term*id *id
expr  expr + term | term
term  term * id | id


Is term  term*id a
handle of term*id*id
at position ε?
Answer: Yes! It brings sentential form
term*id*id into term*id which is clearly
derivable: expr  term  term*id
Spring 16 CSCI 4430, A Milanova
10
Model of an LR parser
Input:
Stack:
State
Grammar
Symbol
a1
ai
…
an
…
$$
LR Parser
sm
Xm
sm-1
Xm-1
…
Parsing table:
s0
action
action[s,a]: Do we shift or reduce?
Spring 16 CSCI 4430, A Milanova
goto
goto[s,A]: After reduction to
nonterminal A, what state is pushed
11
on top of the stack?
id + id*id
Stack
0
0id 3
expr  expr + term | term
term  term * id | id
Input
Action
id+id*id On state 0 and id,
action[0,id] = shift 3
+id*id
On 3 and +, action[3,+] =
reduce by term id
Pop 3 and id, push term.
0term
0term 2 +id*id
etcetera…
On 0 and term,
goto[0,term] = 2
On 2 and +,
action[2,+] = …
12
Model of an LR Parser


Stack is (s0,X1,s1,…Xm,sm), input pointer at ai
action[sm,ai] is shift s



Push ai and state s on stack:
(s0,X1,s1,…Xm,sm,ai,s)
Advance input pointer
action[sm,ai] is reduce by A  β


Pop β (i.e., pop 2*|β| things off the stack - all
symbols in β plus all their corresponding states):
(s0,X1,s1,…Xm-|β|,sm-|β|)
Push A and goto[sm-|β|,A]=s on top of the stack:
13
(s0,X1,s1,…Xm-|β|,sm-|β|,A,s)
Lecture Outline

Bottom-up (LR) parsing

Model of the LR parser

LR Items
Characteristic Finite State Machine
SLR(1) parsing table
LR Parsing variants



Spring 16 CSCI 4430, A Milanova
14
LR Items

start  expr
expr  expr + term | term
term  term * id | id
An LR item is a production with a dot at some
position on the right-hand side




E.g., A  α•β
We are trying to find an A
We already have seen α (it is on top of the stack)
We are looking for β
state 0: start  •expr
expr  •expr+term
expr  •term
term  •term*id
term  •id
Spring 16 CSCI 4430, A Milanova
state 1: start  expr•
expr  expr•+term
Transition on expr
15
Closure of an LR Item



The closure of an LR item A  α•β is the set
of LR items formed as follows:
A  α•β is in the closure of A  α•β
If the dot is in front of a nonterminal B for some item
in the closure, then all of B  •γ1, B  •γ2,… B 
•γn are in the closure (B  γ1, B  γ2,… B  γn are
all productions for B)
Spring 16 CSCI 4430, A Milanova
16
Example

start  expr
expr  expr + term | term
term  term * id | id
Compute closure of start  • expr
Answer:
start  • expr
expr  • expr + term
expr  • term
term  • term * id
term  • id
Spring 16 CSCI 4430, A Milanova
17
Question

start  expr
expr  expr + term | term
term  term * id | id
Compute closure of expr  expr + • term
Answer:
expr  expr + • term
term  • term * id
term  • id

Spring 16 CSCI 4430, A Milanova
18
Question

start  list
list  prefix ;
prefix  prefix , id | id
Compute closure of start  • list
Answer:
start  • list
list  • prefix ;
prefix  • prefix , id
prefix  • id

Spring 16 CSCI 4430, A Milanova
19
Collection of Sets of LR Items with
start  expr
Transitions
expr  expr + term | term
term  term * id | id
0
start  •expr
expr  •expr+term
expr  •term
term  •term*id
term •id
id
expr
1
start  expr•
expr  expr•+term
4
+
expr  expr+•term
term  •term *id
term  • id
term
term
3
term  id•
2
expr  term•
term  term•*id
id
Spring 16 CSCI 4430, A Milanova
6
expr  expr+term•
term  term•*id
*
*
5
term  term*• id
id
7
term  term* id•
20
Example
start  list
list  pre ;
pre  pre , id | id

Construct the collection of sets of LR items
with transitions for the above grammar
Spring 16 CSCI 4430, A Milanova
21
Lecture Outline

Bottom-up (LR) parsing

Model of the LR parser

LR Items
Characteristic Finite State Machine
SLR(1) parsing table
LR parsing variants



Spring 16 CSCI 4430, A Milanova
22
Characteristic Finite State Machine (CFSM)
0
start  •expr
expr  •expr+term
expr  •term
term  •term*id
term •id
id
term  id•
expr
1
start  expr•
expr  expr•+term
+
expr  expr+•term
term  •term *id
term  • id
term
term
3
4
6
expr  expr+term•
term  term•*id
2
expr  term•
term  term•*id
id
*
*
5
term  term*• id
id
7
term  term * id•
The collection of sets of items with transitions is a DFA. This
DFA is one part of the CFSM (we will see the other part shortly).
CFSM states are parsing states. Transitions on terminals
represent shifts. Transitions on nonterminals represent gotos. 23
CFSM
0
start  •expr
expr  •expr+term
expr  •term
term  •term*id
term •id
id
term  id•
expr
1
start  expr•
expr  expr•+term
+
expr  expr+•term
term  •term *id
term  • id
term
term
3
4
6
expr  expr+term•
term  term•*id
2
expr  term•
term  term•*id
id
*
*
5
term  term*• id
id
7
term  term* id•
• 3,7 contain only items of kind A  α•, i.e., reduce items
• 0,4,5 contain items of kind A  α• aβ , i.e., shift items
• 1,2,6 contains both reduce and shift items
24
Question
When the parser is in state 2:
2
expr  term•
term  term•*id
should it reduce by expr  term,
or should it shift * continuing to look for *id ?
Answer: It depends on the lookahead! If
what comes next is + or $$, then reduce.
If it is a *, then shift.
25
start  expr $$
expr  expr + term | term
term  term * id | id
“Reduce by” Labels


For every state that contains a reduce item
A  α•, add label
“reduce by A  α on FOLLOW(A)”
For example, we add label on state 2:
2
expr  term•
term  term•*id
reduce by expr  term on $$,+.
Spring 16 CSCI 4430, A Milanova
26
CFSM
0
start  •expr
expr  •expr+term
expr  •term
term  •term*id
term •id
expr
1
start  expr•
expr  expr•+term
3
term  id•
+
expr  expr+•term
term  •term *id
term  • id
term
term
id
4
6
expr  expr+term•
term  term•*id
2
expr  term•
term  term•*id
id
*
*
5
term  term*• id
id
7
term  term* id•
Add “reduce by A  α on FOLLOW(A)”
State 1: “accept on $$”
State 2: “reduce by expr  term on $$,+”
State 3: “reduce by term  id on $$,+,*”
State 6: “reduce by expr  expr+term on $$,+”
State 7: “reduce by term  term*id on $$,+,*”
27
CFSM

The CFSM has 2 parts



The collection of sets of LR items with transitions
The “reduce by” labels
To construct the CFSM for a grammar G


First, construct the collection of sets of LR items
with transitions
Second, add the “reduce by” labels
Spring 16 CSCI 4430, A Milanova
28
Group Exercise
start  expr
expr  expr + expr | id

Construct the CFSM for above grammar


First, construct collection of sets of LR items
with transitions
Second, add “reduce by” labels
Spring 16 CSCI 4430, A Milanova
29
Lecture Outline

Bottom-up (LR) parsing

Model of the LR parser

LR Items
Characteristic Finite State Machine
SLR(1) parsing table
LR parsing variants



Spring 16 CSCI 4430, A Milanova
30
From CSMR to SLR(1) Parsing Table
1. expr  expr + term
2. expr  term
state
0
1
2
id
7
*
$$
shift 3
shift 4
reduce 2 shift 5
3
4
5
6
+
3. term  term * id
4. term  id
White – action table
Blue – goto table
expr
term
1
2
accept
reduce 2
reduce 4 reduce 4 reduce 4
shift 3
shift 7
6
reduce 1 shift 5
reduce 1
reduce 3 reduce 3 reduce 3
31
SLR(1) Parsing Table
Input: An augmented grammar G’ (G with starting production start  …)
Output: Functions action and goto for G’
Construct C = {I0,I1,…In} the collection of sets of LR items with transitions
State i is constructed from Ii . The parsing actions for state i are
a) If item A  α•aβ is in Ii and there is a transition from Ii to Ij on a, then
set action[i,a] to “shift j”
b) If item A  α• is in Ii then set action[i,a] to “reduce by A  α” for all
terminals a in FOLLOW(A)
c) If start  …• is in Ii then set action[i,$$] to “accept”
The goto transition for state i are constructed for all nonterminals A using
the rule: If there is transition from Ii to Ij on A, set goto[i,A]=j
If the table contains no multiply-defined entries,
the grammar is said to be SLR(1)
32
Conflicts in SLR(1): Shift-reduce

Shift-reduce conflict in state k on terminal a:
State k contains LR item A  β• and a is in
FOLLOW(A)
and
 State k contains item A’  α•aβ’


The parser does not know whether it is at the end
of production A  β and thus must reduce by
A  β, or it is in the middle of production
A’  αaβ’ and thus should shift a and continue
looking for β’
Spring 16 CSCI 4430, A Milanova
33
Conflicts in SLR(1): Reduce-reduce

Reduce-reduce conflict in state k on terminal a:

State k contains item A  β• and a is in FOLLOW(A)
and
 State k contains item A’  β’• and a is in FOLLOW(A’)


The parser does not know whether it is at the end of
production A  β and thus should reduce by A  β, or it
is at the end of production A’  β’ and thus it should
reduce by A’  β’
Indicates a serious problem with the grammar
Spring 16 CSCI 4430, A Milanova
34
LR Parsing Variants: LR(0), SLR(1)

An LR(0) parser does not look at input, 0 lookahead


No state in the CFSM can contain both reduce and shift
items
An SLR(1) parser looks at 1 token of lookahead



This is the variant we studied in class
Resolves certain shift-reduce conflicts by looking ahead
at input a and allowing reduction A  β only if a is in the
FOLLOW set of A
Cannot resolve shift-reduce conflicts such as: A  β•
where terminal a is in FOLLOW(A) and A’  α•β’ where a
in FIRST(β’)
Spring 16 CSCI 4430, A Milanova
35
LR Parsing Variants: LALR(1), LR(1)

LALR(1)




Constructs local, context-sensitive FOLLOW sets
and avoids more conflicts than SLR(1)
An efficiency hack
Most common parsers in practice
LR(1)


Uses a different set of LR items
More states in CFSM automaton allows LR(1) to
keep paths disjoint
Spring 16 CSCI 4430, A Milanova
36
A Hierarchy of Grammars

LL(0) < LL(1) < LL(k)

LR(0) < SLR(1) < LALR(1) < LR(1) < LR(k)

Also, LL(k) < LR(k)
Spring 16 CSCI 4430, A Milanova
37
Next class

Conclude with parsing!

Logic programming and Prolog
Spring 16 CSCI 4430, A Milanova
38