lex bug (v7, 2.8, 4.1, maybe others)

[Sam Kendall]

> Lex versions:  version 7, Berkeley 2.8 and 4.1 (probably
> won't be visible on 32-bit machines, however).
>
> This bug is not in lex itself, but in [/usr/lib/lex/ncform,]
> the boilerplate lex pulls out of its library to make lex.yy.c.
> The original cast two pointers to type int before
> comparing them.  Needless to say, on our 16-bit int pdp11
> this causes errors when the addresses creep past 0x8000.
> I don't know why the casts were inserted, the pointers were
> the same type.

Mike Lesk gave me an explanation for the casts, which if I understood
correctly is as follows: yyt points to the array yycrank[] of size (say)
N.  The program wants to store an extra bit of information in those
pointers; it does so in a strange way, by subtracting N from the
pointer, so that it points to (imaginary) storage below the array.  The
test to see if the pointer points below the array should be just
	yyt < yycrank
(that the test is really a > is presumably an irrelevant detail).
Unfortunately, on some machines
	yyt - N
wraps around the address space, and tests as a very high pointer.  So
the pointers are cast to int, so that the comparison is signed, and very
high pointers test as less than yycrank.

A question to John Stewart: have these casts actually caused errors on
your PDP-11?  yycrank is an external array, and should be in low
memory, so that the problem of addresses past HIGH_ADDRESS/2 should not
occur.

	Sam Kendall	{allegra,ihnp4,ima,amd70}!wjh12!kendall
	Delft Consulting Corp.	    decvax!genrad!wjh12!kendall