Incremental dumps using cpio: BAD NEWS

[TepperL]

The problem is, cpio just doesn't do what you really NEED to properly
restore a disk volume that's been destroyed.  In particular, it only
copies files that have been changed.  It does not keep track of files
that have been deleted or that have had their path changed.

Consider a hierarchy of dump tapes:

	 A level 0 dump that contains all of the files on a
	 particular file system.

	 A level 1 dump that contains the files on a particular
	 file system that have changed since the last level 0
	 dump.

	 A level 2 dump that contains the files on a particular
	 file system that have changed since either the last
	 level 0 or level 1 dump, whicever is more recent.

A nice way to organize dumps is:

	1)  Do a level 0 dump.  Lets say that this used 3 tapes.
	
	2)  Each day, do a level 2 dump.  Typically, each day's dump
	    will require a little more tape than the previous day's.

	3)  If the level 2 dump tape overflows to a 2nd tape, schedule
	    a level 1 dump for the next day.  After that, go back to
	    doing your daily level 2 dumps.

	4)  When the level 1 dump overflows to 3 tapes, schedule a level
	    0 dump for the next day, and then return to the daily level
	    2 dumps.

The problem with cpio is that it doesn't remove files that get deleted.
All of those old objects you and your cohorts created (and later deleted),
old nohup.out files, netnews articles that were saved and printed, but
didn't delete for a week or so, these files make it onto your level 0 and
level 1 dumps.

A particularly insidious problem is RENAMING a directory.  A level 1
dump could contain a bunch of files like /usr/proj/src.  If that directory
gets renamed to /usr/proj.new/src, and a LOT of files in it get put
onto a level 2 tape, then when you restore you'll end up with TWO directories
and a lot of files between them that really are nothing but "restoration
clones".

Don't laugh.  One of the systems here crashed a disk, and the file system
ran out of space when it was being restored for just this reason.  It
took a couple of days to sort it out.

One other thing: If your backup script is the equivalent of:

	find /filesys -mtime xxx -print | cpio -o ...

your cpio tape will not back up files that have had their ownership or
permissions changed.  Those things matter too!  It should look like:

	find /filesys -mtime xxx -o -ctime xxx -print | cpio -o ...

What's really sad is that Version 7 UNIX had a set of programs (called
dump, restor and dumpdir) that had all this stuff figured out.  When
you restored a full volume, what you were left with was an image of what
you had on the last day that you did a back up.  Files would probably
be allocated to different blocks, but they were all there, including
device nodes, permissions, ownerships.  The basic method was:

	mkfs a new file system
	restore level-0 dump
	restore level-1 dump
	restore level-2 dump

It was excellent at restoring full volumes, adequate at restoring a small
number of files (say, 10) and poor if you wanted to restore a lot of files
(such as, everything in a particular directory).  It WAS possible though.

Why wasn't it carried forward to System V?
-- 
Larry Tepper	    {ihnp4 | allegra}!drutx!druil!lat		+1-303-538-1759
	Back to you, Walter.