flex bug with |'d patterns and $.

[Greg Lee]

When patterns associated with several start states are or'd with "|",
a "$" in a pattern may cause an incorrect match by a flex-generated
program.

Here is some sample data:
----------------input data-------------
	none
It's as easy as one, two, three.
	first
It's as easy as one, two, three.
	second
It's as easy as one, two, three.
--------------end data-----------------

And here is a program that works correctly when flex'd:
------------program that works---------

%s	FIRST SECOND
%%

<SECOND>"two" |
<FIRST>"one"  |
"three."		printf("<changed>");

"first"		printf("State 1"); BEGIN FIRST;

"second"	printf("State 2"); BEGIN SECOND;

%%
main() {  yylex();  }
----------end of program that works-----

>From the above input and the above program, comes this output:
-------output of program that works-----
	none
It's as easy as one, two, <changed>
	State 1
It's as easy as <changed>, two, <changed>
	State 2
It's as easy as one, <changed>, <changed>
----end of output of program that works-----

But change the program by adding "$" to the pattern for start
state 0:
---------program that doesn't work---------

%s	FIRST SECOND
%%

<SECOND>"two" |
<FIRST>"one"  |
"three."$		printf("<changed>");

"first"		printf("State 1"); BEGIN FIRST;

"second"	printf("State 2"); BEGIN SECOND;

%%
main() {  yylex();  }
----end of program that doesn't work-------

You'd think the output would be the same, but instead you get:
-----output of program that doesn't work---
	none
It's as easy as one, two, <changed>
	State 1
It's as easy as <changed>wo, <changed>
	State 2
It's as easy as one, <changed>hree.
--end of output of program that doesn't work---

(Lex doesn't understand this sort of construction at all.)

	Greg, lee at uhccux.uhcc.hawaii.edu