A variant of the streams idea

[Arthur M. Keller]

In article <376 at ncr-sd.UUCP> greg at ncr-sd.UUCP (Greg Noel) writes:
>In article <964 at brl-tgr.ARPA> gwyn at brl-tgr.ARPA (Doug Gwyn <gwyn>)
>claims that there are
>>significant differences between random-access
>>(disk) files and sequential (communication)
>>files.  To force random files into the stream
>>model would require sacrificing some of their
>>desirable properties (seekability, sharability,
>>speed), alas.
>
>I'm prepared to believe that the current implementation of streams makes
>this difficult or impossible, but I'm not prepared to believe that the
>idea itself is difficult or impossible.

Some people consider random access to be "stream access with
repositioning".  That might work well in a single-user environment but
fails badly in a multi-user environment with concurrency control.  In
a file system or a database system, the granularity of locking is an
important concept.  It corresponds to the granularity of sharing.  It
may but need not correspond to the granularity of access, although the
two must be comparable (equal or one contained in the other for all
practical purposes).

In streams, the granularity of access is not a very well defined
concept.  You don't really want to lock the "next 100 bytes I will
next read".  Rather, you probably want to lock the "next group of
related fields" commonly called a record.

The notion of streams generally indicates one source and one consumer
for each dataflow pathway, although there may or may not be a
separate, often implicit reverse pathway.  The notion can be
generalized to multiple producers and consumers, but still obeying a
FIFO (or priority queue) discipline.  No notion of modification of the
data in a stream exists, although a transformer may be interposed that
consumes a stream and produces another distinct stream by a
transformation of the consumed stream.  In a randomly accessed, shared
file system, the data do not follow anything resembling a FIFO
discipline.  The data are not consumed, they are referenced; they may
also be created, updated, and destroyed.  To consider a shared file to
be a stream actually means that the file is encapsulated in a process
and your stream is communicating with that process.  Then the process
uses a more traditional discipline to interact with the file.  This
would involve a protocol transformation and the attendant overhead.

I've only briefly touched on some of the issues, but I hope that this
can give the readership of this newsgroup a feeling for the problems
involved.

Prof. Arthur M. Keller
The University of Texas at Austin
Department of Computer Sciences

-- 
------------------------------------------------------------------------------
Arpanet: ARK at SALLY.UTEXAS.EDU
UUCP:    {gatech,harvard,ihnp4,pyramid,siesmo}!ut-sally!ark