buffer i/o using read(2) on BSD sockets

[Kenneth Almquist]

aperez at cvbnet.UUCP (Arturo Perez x6739) writes:
> From article <637 at lot.ACA.MCC.COM>, by ables at lot.ACA.MCC.COM (King Ables):
>> I've been using read(2) to read data from a socket and am having
>> problems when the buffers get large.  I'm hoping someone has run
>> into this before and knows what I am doing wrong.
>>
>> I want to read an arbitrarily large block of data and not have to worry
>> about message boundaries (hence read/write rather than send/recv).
>
> This is one of my pet peeves about BSD sockets.  There is no way
> to read an arbitrary amount of data from a socket.  You have to be
> aware of the kernel level buffering NO MATTER WHAT LEVEL your writing
> your code at; i.e. apps, system, etc.
>
> Why can't the kernel block your process until you get all the data you're
> asking for (unless, of course, FIONBIO or O_NDELAY is set)?  If I'm
> willing to wait, I'm willing to wait.  And if the connection goes down during
> the transfer, I can live with that, too, just return an error.
>
> Why was such a silly decision made?

I presume that the idea of having the read system call return a short
count originally appeared in UNIX to deal with terminal input.  When a
program issues a read system call on a terminal, the read call will
return as soon as a line of input is available, even if the number of
characters in the line is smaller than the size of the buffer passed
to read.  If UNIX did not work this way, most interactive programs
would have to issue a separate read system call for each character,
and stop issuing system calls when a newline character was read.  This
would be inefficient.

When a program issues a read system call on a pipe, the read call will
return as soon as data is available, even if the number of characters
available is smaller than the size of the buffer passed to read.  If
UNIX did not work this way, bc (which opens a pipe to dc) would not
work unless dc inefficiently issued a separate read system call for
every character.

Berkeley sockets intentionally copied the pipe semantics, so that
pipes could be implemented as a special case of sockets.  And the
semantics of Berkeley sockets can be justified independently of this.
If a read on a socket worked the way that King suggests, then rlogin
would have to issue a separate read system call for every character
received from the remote host, which would be very inefficient.

If you have to read a specific number of characters under UNIX, there
are two ways to do it.  One is to place a loop around the read system
call.  The other is to use the fread routine and let the standard I/O
library take care of the buffering.
					Kenneth Almquist