How do I unbuffer the standard output of a child process?

[Jonathan I. Kamens]

  (Hey, Steve, isn't it about time this got added to the FAQ posting? 
Something like this (corrections welcomed)....)

Q) How do I unbuffer the output of a subprocess running on the other
   end of a pipe in my C program?

   Most implementations of the stdio library decide how to buffer the
   stdout stream based on whether or not it is a terminal.  If stdout
   is a terminal, then output is line-buffered, which means that
   output is actually sent to the terminal after each newline is
   printed by the program.  However, if stdout is not a terminal, then
   a larger buffer is used (usually, it is BUFSIZ characters large;
   BUFSIZ is defined in <stdio.h>).

   It is sometimes necessary, however, to force a process to write its
   output after every line, even when it isn't running on a terminal.
   For example, if you are using a standard filter such as "tr" in a
   subprocess, with both the input and output of the filter connected
   to pipes you control, then you might want each line passed to the
   filter to be output by it immediately so that you can use the
   result.

   Unfortunately, this is not easy.  Here are some proposed solutions
   that *will not* work, just to give you an idea of what the problem
   is:

   1. Use setbuf() to disable buffering in the subprocess before
      exec()ing the filter.

      Won't work because all of the stdio initialization takes place
      again in the filter when it starts up, and at that point stdio
      will simply put back the buffering when it realizes that it
      isn't running on a terminal.

   2. Use setbuf() in the parent process on the input pipe to the
      filter, or fflush() in the parent after each chunk of output is
      sent to the printer.

      Doesn't address the problem, which is buffering in the
      subprocess, not buffering in the parent.

   3. Use setbuf() in the parent process on the output pipe from the
      filter.

      Once again, doesn't address the problem.

   If you have access to the source code of the subprocess, then the
   easiest solution to the problem is to modify it to use setbuf() to
   turn off buffering when it starts up, or to fflush() its output
   where you want to be able to read output immediately in the parent
   process.

   If not, then what you have to do is *convince* the subprocess that
   it's talking to a terminal, even though it isn't.  To do this, you
   need to run the subprocess on a pty ("pseudo-tty") rather than
   running talking to it.

   The subject of programming with pty's is too complicated to
   describe in detail here, but the basic idea is:

   1. Scan through the available pty's (usually having names in the
      form "/dev/pty[p-r][0-9a-f]") until you find one you can open.

   2. Fork.

   3. In the child, open the tty corresponding to the pty opened in
      the parent.  For example, if the parent opened "/dev/ptyp5",
      then the child would open "/dev/ttyp5", and replace file
      descriptors 0, 1, and 2 with the file descriptor for the tty.

   4. exec() the filter.

   The man page pty(4), if it exists on your system, may go into more
   detail about pty's.  You might also want to read tty(4).

   If you don't want to learn to work with pty's in C code, then you
   can use Dan Bernstein's "pty" package to do the hard work for you.
   If, for example, your subprocess was originally "tr A-Z a-z", then
   you would use "pty -0 tr A-Z a-z" (assuming that you have installed
   "pty" in your search path).

   For information about obtaining the source code for the pty
   package, see Question 27.

-- 
Jonathan Kamens			              USnail:
MIT Project Athena				11 Ashford Terrace
jik at Athena.MIT.EDU				Allston, MA  02134
Office: 617-253-8085			      Home: 617-782-0710