setpgrp(2) problems on v7, sysIII, sysV, ...

[Andrew Gollan]

I have found what I consider a long standing bug(?) in the implementation
of setpgrp(2) in standard system 3 and system 5 kernels.

If a process is the first to open a tty device and is a process group
leader its process group will become the controlling pgrp for that tty.
t_pgrp will be set to p_pgrp and u_ttyp will be set to &t_pgrp.  When
the process exits the following code cleans up t_pgrp.

	if
	(
	    (p->p_pid == p->p_pgrp)
	    &&
	    (u.u_ttyp != NULL)
	    &&
	    (*u.u_ttyp == p->p_pgrp)
	)
	{
		*u.u_ttyp = 0;
		signal(p->p_pgrp, SIGHUP);
	}


When setpgrp is called the following code only zeros the u_ttyp field
and leaves t_pgrp alone. Hence the terminal remains bound to a pgrp for
ever.

	if (p->p_pgrp != p->p_pid)
		u.u_ttyp = NULL;
	p->p_pgrp = p->p_pid;

I discovered this unpleasantness when writing a new init. I wrote a
program to start up shells, daemons and so forth providing them with
the console as as standard output and error. If the program was not
intended to run interactively on the console I called setpgrp to
dissociate it from the terminal. Nothing thereafter would become the
controlling pgrp of the console. getpass() failed to open /dev/tty when
login finally started, and so on and so forth.

			      Andrew Gollan

UUCP:	seismo!munnari!neology.oz!adjg		ACSnet:	adjg at neology.oz

			Neology Limited, Australia
-- 
			      Andrew Gollan

UUCP:	seismo!munnari!neology.oz!adjg		ACSnet:	adjg at neology.oz

			Neology Limited, Australia