more about pipes in the modern world

[VLD/VMB]

Here is additional information about streams and pipes to add to my
posting of 11-Aug-1986.  It's the result of subsequent discussion
with Dennis Ritchie, who kindly granted me permission to quote him.
I hope this will help those who are trying to implement the full
streams concept (beyond what is in SVR3) to see how it's best done.

I was originally going to summarize Dennis's information, but upon
attempting that, I found that he has, as usual, expressed the ideas
already in about the best possible way, so the following are direct
quotes from DMR.  (I found the bit about implementing FIFOs with
streams particularly instructive, and hope that the AT&T UNIX
product developers in Summit NJ will incorporate this approach in
a future release of UNIX System V, along with the pipes and terminal
handler I previously mentioned.)

[start DMR quotes]

...

Pipes, for example, really do become much cleaner when they are just
cross-connected streams.  SIGPIPE is not a special case; instead it is a
signal generated by writing on any stream whose other end has gone away.
No-delay IO likewise can be handled in the single appropriate place (the
stream read and write routine) instead of being spread around among the
various drivers; so too for select.  Although FIFOs (named pipes) aren't
in V8 proper, I reinstalled them recently in our Cray system, which is SVR2
with ingrafted streams; it added about 10 lines of code, in one place,
and I took the opportunity of stripping out pages of old, special-case
FIFO goo scattered all over the place.

The disengagement of terminal processing from device and network IO was
the most important reason I went into the stream business at all, so the
fact that a separable TTY module hasn't yet made it into SV is genuinely
disconcerting.

...

First, there is, and I think has to be, a real distinction between
stream heads and stream ends; the difference is that some sort of
intelligence (viz., locally-written code that can be executed by a
process) is associated with a stream head, whereas a stream end is a
stupid (or even malevolent) device.  On the other hand, a stream like a
pipe can have two heads.

Second:  all this traffic seems to have resulted from a question about
the detailed fstat behavior of pipes.  In v8, the result of an fstat on a
pipe tells you that it is an ordinary file whose size is always 0.
A seek on any stream file generates ESPIPE.  If there were any ill effects
from this, I don't remember them; it doesn't seem to be an important
issue.

...

Well, here are some details.  The inode table has a new i_sptr slot
that can point to a stream table entry, that in turn connects to the
first of the possibly several stream modules.   The various operations
(read, write, ioctl, open close) check for i_sptr!=NULL.

When a stream device is
opened (they have an extra column in the cdevsw table) i_sptr is filled
in.  This connection, ip->i_sptr, defines a stream head.  The stream
end is the module for the device itself (the end).
This is, I think, true in both V8 and SVr3.

The V8 pipe call allocates two inodes.  Most recently, they are on
a special pipe file system type that has essentially no semantics,
just serves as a label, so they don't have to be put on a real device.
Each inode has a stream head, but the connections are set up so each
head points to the other (this means, incidentally, that pipes are
full-duplex, because there is an independent connection in each direction.)
They have unique inumbers.

Most of the code is symmetrical, and doesn't depend on the difference
between "head" and "end."  Some does.  For example, when a stream is closed,
one starts at the head, popping off processing modules, and stops at
the end.  When a pipe end is closed, though, it is necessary to detect
the crossover and "seal off the other end" by attaching a fake device;
you can't just leave the pointers dangling.

fstat is pretty much transparent to all of this.  Stream devices
look like character devices because they are.  Pipes look like
ordinary files because that is what their inodes are marked as,
but they could be anything; the stream pointer dominates the type
when IO operations occur.

Some side points:  FIFOs on the Cray involve only a single inode
(a real one, created in the usual way SV way).  When it is opened,
the usual stream head is allocated, but it is set up to point to
itself.  Thus they're half duplex like traditional pipes.