token limits in yacc

[Suzanne Artemieff]

Advice?  I'm using yacc (and lex) to build a parser that accepts a 
grammar based on the communications protocol specified in the U.S.
Air Traffic Controller's Manual.  I've run into the problem of a 
(what I consider to be low) limit on the number of tokens that can 
be specified in the yacc input file (127) on the Vax running System
V, version 2 Unix.  I've tried the same input file on a Sun and on a 
68020-based machine with no problem (I currently have about 135 
tokens defined).  My concern is that I will soon hit a tokien limit 
on the other systems, too.  I would prefer not to alter the yacc 
source code (to increase the token limit), as that undermines the 
portability.  Also, I'm not sure what the other ramifications might
entail.  The other alternative that I've considered is to group my 
tokens into classes (such as INDEF_PRONOUN, ADVERB) and then specify 
actions on the yacc rules to call a subroutine to check the vaidity 
of the specific word.  This seems terribly clumsy.  Does anyone have 
any suggestions?  Does anyone know what the token limits are on other 
versions of Unix?  What are the ramifications of upping that limit?

Any help will be greatly appreciated.  You can respond to this group or 
send me email at art at wang.wang.com.