Automatic file compression?

[Robert Barrell]

In article <23653:Oct1019:40:1190 at kramden.acf.nyu.edu>, brnstnd at kramden.acf.nyu.edu (Dan Bernstein) writes:
> It shouldn't be hard to stick this into any filesystem implemented
> outside the kernel. You store all files with a compression type: either
> no compression or a choice of available methods. You keep an MRU cache
> of uncompressed files, including all the files open at the moment. You
> might keep a priority queue of LRU files to switch from uncompressed to
> compressed, or you might have all files compressed immediately upon
> close().

Dan, 
     I understand the gist of what you said, but also realize that such a
filesystem implementation is far beyond my current knowledge of *nix.  Even so,
MUST such a thing be incorporated into a filesystem?  Is it not possible for
appropriate library routines to handle the [un]compression then hand the data
to other, lower-level file i/o routines?  Or is that, by definition, a 
"filesystem implemented outside the kernel?"  
     The whole key to what I want is to eliminate the need for an entire file
to exist in its uncompressed form on the disk at any time.  Rather than taking
the time and disk space to uncompress a file before accessing it, then just
compressing it again when finished,  I'd like to see routines where I'd be able
to say:

Zfseek(fp,1000L,0);
Zgets(string,101,fp);

and have the file position to the 1000th byte of the uncompressed file instead
of the compressed file.  At the moment, when I only need to read information
from compressed files, without having to seek or write, I use:

sprintf(cmd,"zcat %s",filename);
fp = popen(cmd,"r");
...

which works fine.  The problem arises when I want to try to seek, especially if
I wish to seek backwards into the file.

     Your ideas sound wonderful, and such an implementation would mean that
all the regular library commands, dbm functions, etc. could be used directly.
Still, isn't there possibly a simpler interim solution?
     Of course, if anyone out there CAN and DOES implement something which
works either way, I'd like to see it.

-- 
Robert Barrell      |     rbarrell at rdb1.UUCP      | Phillips Consulting Group
Milo's Meadow BBS   | uunet!pcgbase!rdb1!rbarrell | 282 North Shore Drive
login: bbs or nuucp |     "... Pooh just IS."     | Ormond Beach, FL   32176
(904) 441-5028      |     -- The Tao of Pooh      | (904) 672 - 3856