Compressing unix disks

[Barry Shein]

[note that all of this only applies to 4.2 derived systems]

>Doing a 'df' shows that the system reserves 10% of each partition, since
>the amounts in the used and available columns only add up to 90% of the
>total blocks in each partition.  My boss maintains that 10% of the
>AVAILABLE blocks must be kept free, leaving us with only about 81% of the
>total disk space.  I think that the system's already got the space it needs.
>
>Could someone PLEASE tell me I'm right, so we can get back all that wasted
>space?  (9% of 3 Fuji Eagles)
>-- 
>Dave Glowacki           daveg at pwcs.StPaul.GOV         ...!amdahl!ems!pwcs!daveg

>From "A Fast File System for UNIX*", Marshall Kirk McKusick, William
N. Joy, Samaul J. Leffler, Robert S. Fabray.

"In order for the layout policies to be effective, a file system
cannot be kept completely full. For each file system there is a
parameter, termed the free space reserve, that gives the minimum
acceptable percentage of file system blocks that should be free. If
the number of free blocks drops below this level only the system
administrator can continue to allocate blocks."

You're right, the boss is wrong, you can have your 10% back, the
enforced 10% should cover it. But a few things to note:

The 10% figure is a statistical best guess, the algorithm in the file
system only knows whether it's succeeding to get a free block by one
of three progressively less efficient algorithms (in hashalloc()) each
tried in turn. The authors do show in the paper how they arrived at
their 10% figure, it's not a randomly chosen number but is derived
from measurements. This means you *might* get better performance with
even more free space, but chances are also that it's insignificant,
10% is a pretty good margin, if your boss really needs to squeak that
much out then tell him/her/it to buy another disk.

Much more importantly, most of the hit (but not all) occurs when a
block is being allocated, that is, a file is being created and/or
extended on a nearly full file system. You won't lose nearly as much
if you fill up a file system which then is always (or nearly always)
read, such as a partition full of printer fonts or documentation.

The caveat was that as cylinder groups fill disk blocks are likely to
land all over the disk as the last few percent is allocated so there
will likely be some head optimiztion loss on seeks to read blocks.
Then again, the buffer caches can negate this effect if the same stuff
gets read frequently and/or the caches are large enough, big buffer
caches are a big win, forget the extra disk, tell your boss to buy a
little more memory and pump up nbuf...

This all adds up to a wonderful example of what I was told to do as a
student whenever you are asked an operating systems question, always
begin with the phrase "Well, it depends..."

Cheers, and be sure to read Chris Torek's answer also :-)

	-Barry Shein, Boston University