Transcript Lex Fall 2006 Costas Busch - RPI Lex: a lexical analyzer • A Lex program recognizes strings • For each kind of string found the.

Lex
Fall 2006
Costas Busch - RPI
1
Lex: a lexical analyzer
• A Lex program recognizes strings
• For each kind of string found
the lex program takes an action
Fall 2006
Costas Busch - RPI
2
Output
Input
Var = 12 + 9;
if (test > 20)
temp = 0;
else
while (a < 20)
temp++;
Fall 2006
Lex
program
Costas Busch - RPI
Identifier: Var
Operand: =
Integer: 12
Operand: +
Integer: 9
Semicolumn: ;
Keyword: if
Parenthesis: (
Identifier: test
....
3
In Lex strings are described
with regular expressions
Lex program
Regular expressions
Fall 2006
“+”
“-”
“=“
/* operators */
“if”
“then”
/* keywords */
Costas Busch - RPI
4
Lex program
Regular expressions
(0|1|2|3|4|5|6|7|8|9)+
(a|b|..|z|A|B|...|Z)+
Fall 2006
/* integers */
/* identifiers */
Costas Busch - RPI
5
integers
(0|1|2|3|4|5|6|7|8|9)+
Fall 2006
Costas Busch - RPI
[0-9]+
6
identifiers
(a|b|..|z|A|B|...|Z)+
Fall 2006
Costas Busch - RPI
[a-zA-Z]+
7
Each regular expression
has an associated action (in C code)
Examples:
Regular expression
Action
\n
linenum++;
[0-9]+
prinf(“integer”);
[a-zA-Z]+
printf(“identifier”);
Fall 2006
Costas Busch - RPI
8
Default action:
ECHO;
Prints the string identified
to the output
Fall 2006
Costas Busch - RPI
9
A small lex program
%%
[ \t\n]
;
[0-9]+
printf(“Integer\n”);
[a-zA-Z]+
printf(“Identifier\n”);
Fall 2006
/*skip spaces*/
Costas Busch - RPI
10
Output
Input
1234
Integer
Identifier
Identifier
Integer
Integer
Integer
test
var 566
78
9800
Fall 2006
Costas Busch - RPI
11
%{
int linenum = 1;
%}
Another program
%%
[ \t]
\n
; /*skip spaces*/
linenum++;
[0-9]+
prinf(“Integer\n”);
[a-zA-Z]+
printf(“Identifier\n”);
.
Fall 2006
printf(“Error in line: %d\n”,
Costas Buschlinenum);
- RPI
12
Output
Input
1234
Integer
Identifier
Identifier
Integer
Integer
Integer
Error in line: 3
Identifier
test
var 566
78
9800 +
temp
Fall 2006
Costas Busch - RPI
13
Lex matches the longest input string
Example: Regular Expressions
Input:
Matches:
Fall 2006
ifend
if
“ifend”
“if”
Costas Busch - RPI
“if”
“ifend”
14
Internal Structure of Lex
Lex
Regular
expressions
NFA
DFA
Minimal
DFA
The final states of the DFA are
associated with actions
Fall 2006
Costas Busch - RPI
15