How to mess up your tty: (sleep 3;vi)&;vi\n<wait 5 seconds>ZZfg\n

[Phong Vo[drew]]

In article <2997.Jun2704.59.0991 at kramden.acf.nyu.edu>, brnstnd at kramden.acf.nyu.edu (Dan Bernstein) writes:
> If a vi starts up in the background while the tty is in mode X, it'll
> read mode X from the tty before it stops for background output. When you
> get out of the current application and get into vi, it'll make all sorts
> of incorrect conclusions about the tty mode based on its earlier
> reading....
Yes, this is a bad behavior for an application program that expects
to run in the foreground.
> 
> This problem appears to be buried somewhere in curses. Apparently curses
> doesn't try hard enough to make sure it's in the foreground before it
> reads the tty modes. This is true under at least SunOS, Ultrix, and
> every straight BSD machine I can find. I wonder whether it's true under
> SVR4.
As far as I can tell, all vi versions on BSD systems do their own termcap
parsing (via libtermcap.a) and their own ioctls. Only System V vi is based
on <curses>. Even in this case, the dependence on <curses> is only limited
to the terminfo and terminal control stuffs, not the higher level curses
functions that know how to deal with windows. That is, vi only calls setupterm()
to get the initial terminal states, not initscr() which, in addition to calling
setupterm(), also sets up window update stuffs and knows a little about job control.
Part of the reason is that vi does not go well with malloc. Vi also does its own
screen management which is much simpler than the <curses> model. In any case,
because of vi, a fair amount of pain was put into SysV curses so that the first
call to setupterm() will not cause any malloc().
> 
> So where is this bug? How can it be fixed? Whose fault is it, anyway?
> 
It's in the vi code. I would guess in ex.c right at the top of main() when
it tries to grab the terminal states. That's when it ought to check for
foreground/background stuffs.

> I hereby retract my statement from many months back that the only
> available character-mode programs which handle job control correctly are
> (1) my pty program; (2) curses applications, such as vi. (Most other
> examples fall to even the simplest test: rn & then bg then bg then bg.)
I don't know about pty but there isn't any good reason to assume that
curses applications (of which, vi is not!) would handle job control right.
initscr() does a minimal amount of work to check for SIGTSTP but it's just
a function call that could be called anywhere within an application.
I have my own misgiving about that piece of code in initscr() anyway.
In general, I don't like a library to try to be "too smart" and assume to
do too much. Ultimately, it just gets in the way of other pieces of code
and applications have to invent ways to get around such smarty assumptions.
My largest curses-based program is IFS, a language to write
interactive applications with form/menu interfaces (there's a paper
on IFS in IEEE Software, July '90, if anyone is curious). As far as I know,
this program works perfectly with job control. In the course of its
execution, IFS may have to manage several concurrent processes which ocasionally
need to be in foreground, then go back into background. To do this,
it has to do a lot more than the <curses> handling of job control.
Finally, as long as we are talking about signal handling and curses,
if you write curses applications that may occasionally be interrupted
by signals, beware of resulting messy screens. This is because curses is based
on stdio and there is a bug in _xflsbuf() in (as far as I know) all currently
available versions of stdio that causes loss of data if write() is interrupted.
If this event happens during a wrefresh(), certain escape sequences may get lost.
Unfortunately, it's one of those things that application programmers
can do nothing about but screaming at their vendors.