Context-free Languages: Grammars and Automata

CS311 Computational Structures

Context-free Languages: Grammars and Automata Lecture 8

Andrew Black Andrew Tolmach

1

Chomsky hierarchy In 1957, Noam Chomsky published Syntactic Structures, an landmark book that defined the socalled Chomsky hierarchy of languages

original name

language generated

productions:

Type-3 Grammars

Regular

A → α and A → αB

Type-2 Grammars Type-1 Grammars Type-0 Grammars

∩ ∩ ∩

Contex-free

A→γ

Context-sensitive

αAβ → αγβ

Recursively-enumerable

no restriction

A, B: variables, a, b terminals, α, β sequences of terminals and variables 2

Regular languages • • • •

Closed under ∪∩∗·and

⎯

Recognizable by finite-state automata Denoted by Regular Expressions Generated by Regular Grammars

3

Context-free Grammars

4

Context-free Grammars •

More general productions than regular grammars S → w

where w is any string of terminals and non-terminals

4



•


What languages do these grammars generate? S →(A) A → ε | aA | ASA 4



•


What languages do these grammars generate? S →(A) A → ε | aA | ASA

S → ε | aSb

4

Context-free languages more general than regular languages

5


•

{anbn | n ≥ 0} is not regular

5


•


‣

but it is context-free

5


•


‣

•


Why are they called “context-free”?

5


•


‣

•



‣

Context-sensitive grammars allow more than one symbol on the lhs of productions

5


•


‣

•



‣

Context-sensitive grammars allow more than one symbol on the lhs of productions °

xAy → x(S)y can only be applied to the non-terminal A when it is in the context of x and y

5

Context-free grammars are widely used for programming languages

•

From the definition of Algol-60: procedure_identifier::= identifier. actual_parameter::= string_literal | expression | array_identifier | switch_identifier | procedure_identifier. letter_string::= letter | letter_string letter. parameter_delimiter::= "," | ")" letter_string ":" "(". actual_parameter_list::= actual_parameter | actual_parameter_list parameter_delimiter actual_parameter. actual_parameter_part::= empty | "(" actual_parameter_list} ")". function_designator::= procedure_identifier actual_parameter_part.

•

We say: “most programming languages are context-free”

‣ ‣

This isnʼt strictly true … but we pretend that it is! 6

Example

adding_operator::= "+" | "−" .

multiplying_operator::= "×" | "/" | "÷" .

primary::= unsigned_number | variable | function_designator | "(" arithmetic_expression ")".

factor::= primary | factor | factor power primary.

term::= factor | term multiplying_operator factor.

simple_arithmetic_expression::= term | adding_operator term | simple_arithmetic_expression adding_operator term.

if_clause::= if Boolean_expression then.

arithmetic_expression::= simple_arithmetic_expression | if_clause simple_arithmetic_expression else arithmetic_expression.

if a < 0 then U+V else if a * b < 17 then U/V else if k y then V/U else 0

7

Example derivation in a Grammar

•

Grammar:

start symbol is A

A → aAa A→B B → bB B→ε

• A

Sample Derivation: aAa

aaAaa

aaabBaaa

•

aaaAaaa

aaabbBaaa

aaaBaaa aaabbaaa

Language? 8

Derivations in Tree Form A

A

A

a

A

a

A

a

A a

B

A

a a

B

B

b B

B a aa b b ε

A

a a

a

b B ε

9

see more easily what this language is if you think of a as a left parenthesis and b as a right parenthesis ")". =ewed in this way, L(G3 ) is the language of strings of properly nested parentheses.

Arithmetic expressions in a programming language

EXAM PLE

2.4 ........................................................................................................................

Consider grammar G4 = (V, S , R , (EXP R)). V is {(EX PR),(TERM),(FACTOR)}and E is {a, +, x, (, I}. T h e rules are (EXPR)+ (EXP R)+(T ERM1 )(TERM) (TERM) + (TERM)x (FACTOR)1 (FACTOR) (FACTOR)+ ( (EXPR)) 1 a T h e two strings a+ax a and (a+a) xa can be generated with grammar T h e parse trees are shown in the following figure.

•

Derive: a + a × a

10

FIGURE

2.5

11 xa Parse trees for the strings a+axa and (a+a)

Notice how the grammar FIGURE 2.5 gives the meaning a + (a×a) 11 xa Parse trees for the strings a+axa and (a+a)

Grammars in real computing •

•

CFGʼs are universally used to describe the syntax of programming languages

•

Perfectly suited to describing recursive syntax of expressions and statements

•

Tools like compilers must parse programs; parsers can be generated automatically from CFGʼs

•

Real languages usually have a few non-CF bits

CFGʼs are also used in XML DTDʼs 12

Formal definition of CFG •

A Context-free grammar is a 4-tuple (V, Σ, R, S) where 1. V is a finite set called the variables (nonterminals) 2. Σ is a finite set (disjoint from V) called the terminals, 3. R is a finite set of rules, where each rule maps a variable to a string s ∈ (V ∪ Σ)* 4. S ∈ V is the start symbol 13

Definition of Derivation • • •

Let u, v and w be strings in (V ∪ Σ)*, and let A →w be a rule in R, then uAv uwv (read: uAv yields uwv) We say that u

*

v (read: u derives v) if

u = v or there is a sequence u1, u2, … uk, k ≥ 0, s.t. u u1 u2 … uk v

•

The language of the grammar is

{w ∈ Σ* | S * w } 14

Derivations •

Parse Trees

Each derivation can be viewed as a parse tree with variables at the internal nodes and terminals at the leaves

• •

Start symbol is at the root Each yield step uAv

uwv where w=w1w2...wn

corresponds to a node labeled A with children w1,w2,...,wn.

•

The final result in Σ* can be seen by reading the leaves left-to-right 15

Simple CFG examples •

•

Find grammars for:

•

L = {w ∈ {a,b}* | w begins and ends with the same symbol}

•

L = {w ∈ {a,b}* | w contains an odd number of aʼs}

•

L [(ε + 1)(01)*(ε + 0)]

Draw example derivations 16

All regular languages have context free grammars •

Proof:

• •

Regular language is accepted by an NFA.

•

Any regular grammar is also a CFG. (Immediate

We can generate a regular grammar from the NFA (Lecture 6, Hein Alg. 11.11) from the definition of the grammars).

17

Example • • • • • •

S→aQ1

S→bR1

Q1 →aQ1

Q1 →bQ2

Q2 →aQ1

Q2 →bQ2

R1 →aR2

R1 →bR1

R2 →aR2

R2 →bR1

Q1 →ε

R1 →ε

• Resulting grammar may be quite

different from one we designed by hand. 18

Some CFGʼs generate non-regular languages •

•

Find grammars for the following languages

• •

L = {anbman | a,b ≥ 0}

•

L = {wwR | w ∈ {a,b}*}

L = {w ∈ {a,b}* | w contains equal numbers of aʼs and bʼs}

Draw example derivations 19

Ambiguity •

A grammar in which the same string can be given more than one parse tree is ambiguous.

• Example: another grammar for arithmetic expressions ‹EXPR› →

‹EXPR› + ‹EXPR› | ‹EXPR› × ‹EXPR› | ( ‹EXPR› ) |a

• Derive: a + a × a 20

• This grammar is ambiguous: there are two different parse trees for a + a × a

• Ambiguity is a bad thing if weʼre GURE 2.6 interested in the structure of the parse e two parse trees for the string a+axain grammar G5

• Ambiguity doesnʼt matter if weʼre interested only in defining a language. 21 This grammar doesn't capture the usual precedence relations and s

Leftmost Derivations •

In general, in any step of a derivation, there might be several variables that can be reduced by rules of the grammar.

•

In a leftmost derivation, we choose to always reduce the leftmost variable.

•

Example: given grammar S → aSb | SS | ε

•

A left-most derivation:

S

•

aSb

aSSb

aaSbSb

aabSb

aabb

A non-left-most derivation:

S

aSb

aSSb

aSb

aaSbb

aabb 22

Ambiguity via left-most derivations •

Every parse tree corresponds to a unique left-most derivation

•

So if a grammar has more than one leftmost derivation for some string, the grammar is ambiguous

•

Note: merely having two derivations (not necessarily left-most) for one string is not enough to show ambiguity 23

Ambiguity

E GURE 2.6

E⨉E

E+E⨉E

E

E+E

a+E

e two parse trees for the string a+axa in grammar G5 a+E⨉E a+a⨉E a+E⨉E

a+a⨉a

a+a⨉E

a+a⨉a

This grammar doesn't capture the usual precedence relations and so up the before the x or vice versa. In contrast grammar24Ga, gen

+

Context-free languages •

Closed under ∪, ∗ and ·, and under ∩ with a regular language

‣

How do we prove these properties?

•

Not closed under intersection, complement or difference

•

Recognizable by pushdown automata

‣

A pushdown automaton is a generalization of a finite-state automaton 25

Pushdown Automata •

Why canʼt a FSA recognize anbn?

‣ •

input

finite state control

reject

How can we fix the problem?

‣ •

“storage” is finite

accept

add unbounded storage

stack

Whatʼs the simplest kind of unbounded storage

‣

a pushdown stack 26

History •

PDAs independently invented by Oettinger [1961] and Schutzenberger [1963]

•

Equivalence between PDAs and CFG known to Chomsky in 1961; first published by Evey [1963].

27

Executing a PDA •

The behavior of a PDA at any point depends on ⟨state,stack,unread input⟩

•

PDA begins in start state with stated symbol on the stack

•

On each step it optionally reads an input character, pops a stack symbol, and non-deterministically chooses a new state and optionally pushes one or more stack symbols

•

PDA accepts if it is in a final state and there is no more input. 28

Example PDA

he PDA Ati that recognizes {on 1 " n

> 01

29

Example Execution: 0011 Begin in initial state with empty stack


Stack

> 01

30

Example Execution: 0011


$ Stack

> 01

31



0 $ Stack

> 01

32



0 0 $ Stack

> 01

33



0 $ Stack

> 01

34



$ Stack

> 01

35



Stack Accept!

> 01

36

PDAʼs can keep count! •

This PDA can recognize {0n1n | n ≥ 0} by

‣ ‣ ‣

•

First, pushing a symbol on the stack for each 0 Then, popping a symbol off the stack for each 1 Accepting iff the stack is empty when the end of input is reached (and not before)

The size of the stack is unbounded.

‣

That is, no matter how big the stack grows, it is always possible to push another symbol on it.

‣

So PDAʼs can use the stack to count arbitrarily high 37

Pushdown Automata (PDA) •

A pushdown automaton M is defined as a M = (Q, Σ, Γ, δ, q0 , Z0 , F ) where: 7-tuple: ,

‣ ‣ ‣ ‣ ‣ ‣

q0 ∈ Q Q is a set of states, is the start state

Σ is the input alphabet,

Z0 ∈ Γ the initial stack symbol Γ is the stack alphabet, is δ : (Q × Σε × Γε ) → P{Q × Γ } is the transition function ∗

F ⊆ Q is a set of final states, and

Xε = X ∪ {ε} the set X augmented with ε , 38

Executing a PDA •

The configuration (or PDA + Hopcroftʼs instantaneous description, ID) of a PDA consists of 1. The PDA, 2. The current state of the PDA, 3. the remaining input, and 4. The contents of its stack.

39

Transitions •

We defined δ : (Q × Aε × Γε ) → P{Q × Γ } ∗

‣

‣

a, γ) The transitions δ(q, are applicable iff °

q is the current state,

°

, a = ε or the next character on the input tape is a , and

°

γ = ε or the top of the stack is ,

γ

If you select a transition , (q ! , ω) then !

°

q The new state is

°

γ popped off of the stack, and γ != ε is if ,

°

the (possibly empty) sequence of symbols ω is pushed onto the stack 40

PDAs are non-deterministic

For two reasons:

• •

The transition function δ answers a set We have the choice of taking an εtransition, or reading an input symbol

41

Trivial Example

Context-Free Languages and Pushdown Automata Hein Example 12.4

ε!, X ε!, Y Start

s

pop

push(X )

0

push(X)

ε, Y ! nop

f

!

As the example shows, algorithm (12.5) doesn’t always give th s. For PDA to accept state ca δ(s, ε, Y)a=simpler {(0, XY)} δ(0, ε{"} , Y)by= final {(f, Y)} • example, follows:

•

•

δ(0, ε, X) = {(0, ε)}

Start 42

Accepting a String •

There are two ways in which a PDA can accept a string: 1. final state is in the set F (implies F non-empty) 2. if F is empty (there are no final states), then the PDA accepts when the stack is empty

•

The previous example accepts by …

43

Acceptance by Final State A run of PDA M = (Q, A, Γ, δ, q0 , γ0 , F ) is a sequence a0

a1

an−1

(q0 , γ0 ) → (q1 , s1 ) → · · · → (qn , sn )

with q0 , . . . , qn ∈ Q, s1 , . . . , sn ∈ Γ , and a0 , . . . , an−1 ∈ A ∗

such that:

for all i ∈ [0 .. n − 1]. (qi+1 , γi+1 ) ∈ δ(qi , ai , γi ) and

si = γi ti and si+1 = γi+1 ti for some ti ∈ Γ , and ∗

w = a0 a1 a2 . . . an−1 is the input. The run accepts w if qn ∈ F.

The language of is M, L(M ) given by

L(M ) = {w ∈ A | w is accepted by some run of M } ∗

44

Acceptance by Empty Stack •

There is an alternative definition of acceptance:

‣ ‣

If the set of final states is empty, then the PDA accepts an input string if it can read all of the characters of the string and end up in a state in which the stack is empty.

45

Acceptance by Empty Stack A run of PDA M = (Q, A, Γ, δ, q0 , γ0 , ∅) is a sequence a0

a1

an−1

(q0 , γ0 ) → (q1 , s1 ) → · · · → (qn , sn )

with q0 , . . . , qn ∈ Q, s1 , . . . , sn ∈ Γ , and a0 , . . . , an−1 ∈ A ∗

such that:

for all i ∈ [0 .. n − 1]. (qi+1 , γi+1 ) ∈ δ(qi , ai , γi ) and

si = γi ti and si+1 = γi+1 ti for some ti ∈ Γ , and ∗

w = a0 a1 a2 . . . an−1 is the input. The run accepts w if sn = ε.

The language of is M, L(M ) given by

L(M ) = {w ∈ A | w is accepted by some run of M } ∗

46

It doesnʼt matter! •

PDAs that accept by empty stack and PDAs that accept by final state have equivalent power

•

What does this mean?

‣ ‣

That given either one, we can build the other Consequently, the class of languages that they accept are the same

47

Example (Hein 12.2)

push(X)

ε

means that (in state 0), with input a and

with X on the top of the stack, we can transition into state 0, and put an additional X onto the stack. 48

Example Empty stack acceptance

(Hein 12.2)

push(X)

ε

means that (in state 0), with input a and

with X on the top of the stack, we can transition into state 0, and put an additional X onto the stack. 48

Example M = ({0,1}, {a, b}, {X}, δ, 0, X, ∅)

(Hein 12.2)

push(X)

ε

49

Example M = ({0,1}, {a, b}, {X}, δ, 0, X, ∅)

(Hein 12.2)

push(X)

ε

‣ ‣

δ(0, a, X) = {(0, XX)} δ(0, ε, X) = {(1, ε)}

‣

δ(1, b, X) = {(1, ε)}

49

Transformation to a Final State PDA

ε

50


ε

50


ε S

50


ε Start

S

50


ε

_____ Start

push(Y)

S

50


ε

_____ Start

push(Y)

S

F

50


ε

_____ Start

push(Y)

S

F

50


ε

_____ Start

push(Y)

S

F

50


_____

ε, Y push(Y) S push(X) Start

ε F

50


_____


ε, Y

ε

nop

ε, Y nop

F

50


_____


‣

ε, Y

ε

nop

ε, Y nop

F

N = ({0,1, S, F}, {a, b}, {X,Y}, δ´, S, Y, {F})

50


_____


ε, Y

ε

nop

ε, Y nop

F

‣

N = ({0,1, S, F}, {a, b}, {X,Y}, δ´, S, Y, {F})

‣

δ´ = δ + δ´(S, ε, Y) = {(0, XY)} + δ´(0, ε, Y) = {(F, ε)} + δ´(1, ε, Y) = {(F, ε)} 50

Context-Free Languages and Pushdown Automata

Transformation to an EmptyStack PDA ε!, X

push(Y)

Start

ε!, Y s

push(X )

pop

0

ε!, Y

f

!

nop

As the example shows, algorithm (12.5) doesn’t always give th s. For example, a simpler PDA to accept {"} by final state ca follows:

Start 51



push(Y)

Start

ε!, Y s

push(X )

pop

0

ε!, Y

f

!

nop

S


Start 51



push(Y)

Start

ε!, Y s

push(X )

pop

0

ε!, Y

f

!

nop

S


Start 51



push(Y)

Start _____

ε!, Y s

push(X )

push($)

pop

0

ε!, Y

f

!

nop

S


Start 51


Transformation to an EmptyStack PDA ε!, X ε!, Y

Start _____

s

push(X )

push($)

pop

0

ε!, Y

f

!

nop

S


Start 51



Start _____ push($)

S

s

push(X )

ε, $ push(Y)

pop

0

ε!, Y

f

!

nop


Start 51



Start _____ push($)

S

s ε, $ push(Y)

pop

push(X )

0

e

ε!, Y

f

!

nop


Start 51



Start _____ push($)

S

s ε, $ push(Y)

pop

push(X )

0

e

ε!, Y

f

!

nop


Start 51



Start _____ push($)

S

s ε, $ push(Y)

pop

push(X )

0

e

ε!, Y

f

!

nop


Start 51



Start _____ push($)

S

s ε, $ push(Y)

pop

push(X )

0

e

ε!, Y

f

!

nop ε, ε doesn’t

As the example shows, algorithm (12.5) always give th s. For example, a simpler PDA to accept {"} by final state ca follows:

Start 51



Start _____ push($)

S

s ε, $ push(Y)

pop

push(X )

0

e

ε!, Y

f

!


As the example shows, algorithm (12.5) always give th ε, Y s. For example, a simpler PDA popto accept {"} by final state ca follows:

Start 51



Start _____ push($)

S

s ε, $ push(Y)

pop

push(X )

0

e

ε!, Y

f

!


X As the example shows, ε, algorithm (12.5) always give th ε, Y pop s. For example, a simpler PDA popto accept {"} by final state ca follows:

Start 51



Start _____ push($)

S

s ε, $ push(Y)

pop

push(X )

ε, $ pop

0

e

ε!, Y

f

!



Start 51



Start _____ push($)

S

s ε, $ push(Y)

pop

push(X )

ε, $ pop

0

e

ε!, Y

f

!



Start 51

Other Variations ‣

Hein says: “the execution of a PDA always begins with one symbol on the stack”

‣ ‣

Other authors say “the stack is initially empty” Does it matter?

52

Other Variations

•

‣


‣ ‣


Can you always convert an initially empty PDA to one with an initial symbol on the stack?

52

Other Variations ‣


‣ ‣


•

Can you always convert an initially empty PDA to one with an initial symbol on the stack?

•

Can you always convert a PDA with an initial symbol to one whose stack is initially empty? 52

Stack Operations •

Hopcroft et al. says:

‣ ‣

•

you can push zero or one symbols onto the stack

Hein says:

‣

•

you can push on as many stack symbols as you like, including the one that you just popped.

Other authors say

‣

•

on each transition, to look at the top of the stack you must pop it, but

on each transition, you can push, pop or nop the stack, but you canʼt do more than one thing

Does it matter? 53

More Examples L1= {anbn: n≥0} a,ε

b,#

push(#)

pop

_____

ε,$

push($)

nop

ε, ε nop

54

Palindromes of even length L2= {wwR⎮ w ∈ {a, b}*} b,ε a,ε

a,a b,b

push(b) push(a)

pop pop

_____

ε,$

push($)

nop

ε, ε nop

55


a,a b,b

push(b) push(a)

pop pop

_____

ε,$

push($)

nop

ε, ε nop

•

This machine is non-deterministic. (Why?)

55


a,a b,b

push(b) push(a)

pop pop

_____

ε,$

push($)

nop

ε, ε nop

• •

This machine is non-deterministic. (Why?) Can you design a deterministic PDA for L2? 55

Balanced Parenthesis L3= {w ∈ {(, )}* : w is a balanced string of parenthesis}

(,ε

),#

push(#)

pop

_____

ε,$

push($)

nop

56

Equal numbers of a s and b s a,ε

a,b

push(a)

pop

b,ε

b,a

push(b)

pop

_____

ε,$

push($)

nop

57