wall(1) causes window(1) to hang

[Dave Stewart]

Index:	ucb/window/wwiomux.c 4.3BSD

Description:
	Window uses fcntl(2) to set the FASYNC flag on fd 0.  (FASYNC means
	that pending I/O causes a SIGIO to get sent to the process group.  This
	is how window knows that there is keyboard input.)  This causes the
	FASYNC bit to get set in the open file structure flags for fd 0.  Also,
	an fioctl(fp, FIOASYNC, 1) is done to set the ASYNC bit in the t_state
	field of the tty structure (because fioctl() calls ttioctl()).  This
	takes place in fset(), which was called by fcntl()
	(sys/kern_descrip.c).

	Wall comes along and opens each tty in the utmp file in turn and
	sets the FNDELAY flag on the open tty using fcntl().  This does a
	fioctl(fp, FIONBIO, 1) on the fd, as described above, but because
	FNDELAY and FASYNC are in the same flags word, a fioctl(fp, FASYNC, 0)
	call is also made, and the FASYNC flag is cleared in the tty.  Window,
	which is waiting for a SIGIO to tell it to read() from the tty,
	will never get one and has wedged the poor luser's terminal.  You
	now need to login from another terminal and kill your window
	process.
Repeat-By:
	Run window.  In a window, run wall.  The wall output will write to
	your tty and your tty will ignore you from then on.  Note that
	any output bound for a window still gets written, because window
	uses a select() to wait for window output.
Fix:
	There are many ways to change wall and/or window to prevent this
	from happening.  However, the real problem stems from the fact that
	one ought to be able to determine the current value of the
	t_state field of the tty structure without opening /dev/kmem.  In
	any case, my intuition is that the kernel should not allow this
	to happen: if a tty needs to have FASYNC or FNDELAY set by a process,
	this should occur without fouling up the settings that another process 
	has made.  It is easy to blame the implementation of fcntl(), but
	doing a raw FIONBIO or FIOASYNC ioctl() on the tty is almost as bad,
	since there is still no way to tell whether or not you are messing
	up the current settings.  This is an appeal to you subscribers for a 
	better fix.

	In the meantime, I offer the following modification to window which
	is a gross hack, and is not elegant, but it should free up the
	terminal.  Instead of having window sleep in select() forever, I
	have it timeout after SELTIMEOUT seconds, at which time it forces
	FASYNC to be asserted on fd 0.  (Since there is no way to read this
	information besides opening kmem and I don't want to make window
	setuid kmem, thank you, I am forced to set the flag every time - the
	only way I can be *sure* it's set).  The cost is that an idle
	window session will accrue one system call every SELTIMEOUT seconds.
	One more note: There can be no output to any window for SELTIMEOUT
	seconds for the terminal to get unjammed.
	Diffs follow:
RCS file: RCS/wwiomux.c,v
retrieving revision 1.0
retrieving revision 1.1
diff -c -r1.0 -r1.1
*** /tmp/,RCSt1026703	Fri Jul 25 17:59:12 1986
--- /tmp/,RCSt2026703	Fri Jul 25 17:59:13 1986
***************
*** 1,5
  #ifndef lint
! static char *RCSid = "$Header: wwiomux.c,v 1.0 86/07/25 15:44:35 root Exp $";
  #endif
  #ifndef lint
  static char sccsid[] = "@(#)wwiomux.c	3.14 4/24/85";

--- 1,5 -----
  #ifndef lint
! static char *RCSid = "$Header: wwiomux.c,v 1.1 86/07/25 17:54:45 davest Exp $";
  #endif
  #ifndef lint
  static char sccsid[] = "@(#)wwiomux.c	3.14 4/24/85";
***************
*** 13,18
  
  #include "ww.h"
  #include <sys/time.h>
  
  /*
   * Multiple window output handler.

--- 13,20 -----
  
  #include "ww.h"
  #include <sys/time.h>
+ #include <fcntl.h>
+ #define SELTIMEOUT	5
  
  /*
   * Multiple window output handler.
***************
*** 33,38
  	register char *p;
  	char c;
  	static struct timeval tv = { 0, 0 };
  	char noblock;
  
  loop:

--- 35,41 -----
  	register char *p;
  	char c;
  	static struct timeval tv = { 0, 0 };
+ 	static struct timeval tmout = { SELTIMEOUT, 0 };
  	char noblock;
  
  loop:
***************
*** 63,68
  			return;
  		}
  	}
  	wwnselect++;
  	n = select(wwdtablesize, &imask, (int *)0, (int *)0,
  		noblock ? &tv : (struct timeval *)0);

--- 66,72 -----
  			return;
  		}
  	}
+ doselect:;
  	wwnselect++;
  	n = select(wwdtablesize, &imask, (int *)0, (int *)0,
  		noblock ? &tv : &tmout);
***************
*** 65,71
  	}
  	wwnselect++;
  	n = select(wwdtablesize, &imask, (int *)0, (int *)0,
! 		noblock ? &tv : (struct timeval *)0);
  	wwsetjmp = 0;
  
  	if (n < 0)

--- 69,75 -----
  doselect:;
  	wwnselect++;
  	n = select(wwdtablesize, &imask, (int *)0, (int *)0,
! 		noblock ? &tv : &tmout);
  	wwsetjmp = 0;
  
  	if (n < 0)
***************
*** 70,76
  
  	if (n < 0)
  		wwnselecte++;
! 	else if (n == 0)
  		wwnselectz++;
  	else
  		for (w = wwhead.ww_forw; w != &wwhead; w = w->ww_forw) {

--- 74,80 -----
  
  	if (n < 0)
  		wwnselecte++;
! 	else if (n == 0) {
  		wwnselectz++;
  		if (!noblock) {		/* timeout, FASYNC may have cleared */
  			fcntl(0, F_SETFL, wwnewtty.ww_fflags|FASYNC);
***************
*** 72,77
  		wwnselecte++;
  	else if (n == 0)
  		wwnselectz++;
  	else
  		for (w = wwhead.ww_forw; w != &wwhead; w = w->ww_forw) {
  			if (w->ww_pty < 0 || (imask & 1 << w->ww_pty) == 0)

--- 76,87 -----
  		wwnselecte++;
  	else if (n == 0) {
  		wwnselectz++;
+ 		if (!noblock) {		/* timeout, FASYNC may have cleared */
+ 			fcntl(0, F_SETFL, wwnewtty.ww_fflags|FASYNC);
+ 			wwsetjmp = 1;
+ 			goto doselect;
+ 		}
+ 	}
  	else
  		for (w = wwhead.ww_forw; w != &wwhead; w = w->ww_forw) {
  			if (w->ww_pty < 0 || (imask & 1 << w->ww_pty) == 0)
-- 
David C. Stewart                          uucp:    tektronix!davest
Unix Systems Support Group                csnet:   davest at TEKTRONIX
Tektronix, Inc.                           phone:   (503) 627-5418