Record-access libraries (Was: Re: VMS vs. UNIX file system)

[Eric S. Raymond]

In article <207 at cvbnet2.uucp>, aperez at cvbnet2.UUCP (Arturo Perez Ext.) writes:
> What I'm curious about is the fact that I've never heard of any record
> access libraries for Unix.  I know that I've written simpleminded record
> access applications.  I'm sure other people have as well.  Is there anyone
> actually selling record access libraries for the Unix community?  If not
> why isn't anyone doing it?

Shortest answer: because it's not worth doing.

Medium-length answer: because, for the most common case of sequentially-
accessed fixed-length records, 'record access' under UNIX is adequately
handled from C by fread(3)/fwrite(3) of a struct buffer. The less common cases
are traditionally handled by ASCII encoding with a \n-terminated line
structure, the one kind of variable-length 'record' for which there is strong
support under UNIX.

Long answer: well, what does a record-access library *do*, anyhow, that byte-
stream access doesn't give you? There are basically, two possible answers:

    1. Encourage code simplification by provision of 'higher-level' primitives.
    2. Access speed or space optimization.

Both of these turn out to be mostly illusion. UNIX's byte-stream primitives
are sort of ultimately simple, certainly a lot simpler than the thicket of RMS
entry points. The text-file support in stdio is pretty clean and basic too
(which is why it's become a de-facto standard for C implementations even on
non-UNIX systems).

A possibility of significant speed or space optimization over stdio or simple
fread/fwrite only enters if you're talking about access to variable-length
records that can't reasonably be represented in ASCII. Such applications tend
to a) require indexed access, in which case you're already talking database
and the 'record library' likely has to be custom to handle housekeeping info,
or b) be sufficiently speed or space-critical that no savvy programmer is
really going to like someone else's packaged library for access.

And don't forget that at some level the record-access has to be doing
byte-stream I/O *anyway*. The only way it can win is by buffering. And wouldn't
you rather tune your own buffer sizes?

Record-access libraries sound like a decent idea, but in practice they tend to
introduce a lot of interface complexity (which makes your code ugly) and
premature optimization (which actually hurts performance). VMS's RMS could
stand as a perfect example of both these lossages.
-- 
      Eric S. Raymond                     (the mad mastermind of TMN-Netnews)
      UUCP: ...!{uunet,att,rutgers}!snark!eric = eric at snark.UUCP
      Post: 22 S. Warren Avenue, Malvern, PA 19355      Phone: (215)-296-5718