Does NEWLINE always flush stdio buffer?

[Steve Summit]

In article <11012 at ihlpl.ATT.COM> knudsen at ihlpl.ATT.COM (Knudsen) writes:
>I'd always understood that printf'ing any string ending in '\n'
>(newline) would flush the I/O buffer.
>However, I've found that this will not always flush if stdout has been
>redirected to a pipe.
>
>I know about fflush(), which always works, but it can be a pain
>to call it in some cases.

Some will disagree, but I believe it is best to call fflush
explicitly whenever the output should be seen right now, such as
after prompts, and after intermediate output that precedes long
compute-intensive activity.

Others have also found fflush inconvenient, which is why various
special cases have been added in attempt to obviate its use, such
as that a newline can flush output, as can a pending read
(actually, getchar or the equivalent) on stdin.  (One or both of
these behaviors is often referred to as "line buffering.")  The
trouble is, as you have already discovered, that line buffering
is not sufficient, among other things because it applies, for
efficiency reasons, only if isatty(fileno(stdout)), and not if
stdout is a pipe.

(Normally I deprecate counter-functional design decisions based
solely on efficiency concerns, but in this case the decision is
correct: if line buffering is to be turned on implicitly at all,
it must be done so only for interactive invocations, and isatty
is the closest convenient thing we have for determining
interactivity.  Stdio buffering exists in the first place at
least partially for efficiency reasons: doing low-level,
system-call I/O on a character-at-a-time basis is notoriously,
and truly, inefficient, i.e. it does make a difference in
practice.  Line buffering partially removes the advantages of
full buffering; it must not be turned on implicitly when
batch-style jobs are generating large quantities of output
redirected to a file.)

I have rarely found it inconvenient to call fflush explicitly --
if a program asks the user more than a couple of questions, I
usually have a centralized prompt-and-get-answer routine, and I
can place a single call to fflush there.  For cases other than
prompts, there is usually one place within a main processing loop
somewhere where an fflush can be added to make sure any pending
output has been displayed.

It is possible to turn off buffering entirely, if line buffering
is inadequate and explicit fflush'es are somehow unworkable,
although to do so is worse from a performance standpoint than not
using stdio at all.  That is, an unbuffered printf("Hello, world!\n")
is even worse than write(1, "Hello, world!\n", 14) because the
former will typically generate fourteen calls to write(), each
passing one character.  (This is why programs like tar with the v
option perform poorly on heavily-loaded machines or over network
connections; tar's verbose output is printed to stderr which is
normally unbuffered.)

It is rather straightforward to put special cases into the stdio
code so that printf and the like on unbuffered streams call
write() with as many characters as are available, without doing
any buffering.  Unfortunately, I have not seen a distributed
version of stdio which does so.

                                            Steve Summit
                                            scs at adam.pika.mit.edu

P.S.  Note that it is unclear to talk about "the stdio buffer;"
there is typically one buffer per open stream: stdin, stdout, and
any fopen'ed streams.