saving directories when they don't fit on 1 tape

[Brian Rice]

In article <1454 at mdbs.UUCP> wsmith at mdbs.UUCP (Bill Smith) writes:

>Is there an automatic, reasonably portable way to save a directory and all 
>files beneath it on tape when it does not fit on a single piece of media 
>and can not easily be split at the next level down in the directory tree?

Use cpio.  First, there are some cpio's which support spreading
tape files across several tapes.  Ours does, I'm confident; check 
your cpio's man page to make sure.  But I don't think vanilla 
System V's do (or are required to by AT&T), and SunOS 4.0's cpio 
man page says it doesn't.

So suppose your cpio doesn't do the trick by itself.  We can use
cpio's flexibility to work around dern near anything.  Here's a 
simple, *approximate* method, which will work most of the time and
is hassle-free:  Suppose you have 400 files which take up 190 
megabytes, which you want to put on two 150-megabyte tapes.  Just 
put in a tape, give this command (as root):

find directory_to_write -print | head -200 | cpio -vBo >tape_device

Then put in a new tape, and give this command:

find directory_to_write -print | tail -200 | cpio -vBo >tape_device

When you reload these using cpio -i, be sure to include the -d option,
which creates directories as needed.  Depending on your medium, you
might need to use some other option than B (see the man page; there's
scads of them).

Obviously, this method will fail if one file occupies, say, 160 MB
and the remaining 399 together only take 30 MB. So, if you want a 
method which will work all the time, you're going to have to dress 
this up with a shell script.  The idea is to partition all the files 
into groups, each of which has total size less than one of your 
physical media, then cpio each group out, prompting the operator 
between each group to insert a new tape.  If I were going to write this,
here's the strategy I'd use:

1.  Use find to create a list of all files which you wish to 
    write to tape.  Call it /tmp/out1.
2.  Pipe this file to a program which, for each line of input,
    runs /bin/ls -sd on it, and appends the result to a file.
    Call this file /tmp/out2.
3.  awk '{print $1}' /tmp/out2 > /tmp/out3.
4.  Run a program which proceeds through /tmp/out1 and /tmp/out3, 
    adding the value of each line of /tmp/out3 to a cumulative total 
    while appending each corresponding line of /tmp/out1 to /tmp/out4.
    When we find that adding in the value of a new line of /tmp/out3 to
    our total would cause the total to exceed tape capacity (less a 
    fudge factor), or when we reach the end of file on /tmp/out1 and
    /tmp/out3, we do this command:

    cat /tmp/out4 | cpio -ovB > tape_device

    If there are no more lines in /tmp/out1 and /tmp/out3, we're
    done.  Otherwise:

    echo Please insert a new tape and press a key...
    # Wait for the user to press a key, using your favorite method.

    Then we empty out /tmp/out4 and start reading from /tmp/out1 and
    /tmp/out3 again.
5.  rm /tmp/out[1234]

Pretty ugly, ain't it.  That's why we should hide it in a shell script.
But I think it'll work.  /tmp/out[123] should all wind up with the same 
length, unless something is dreadfully wrong (that -d option on ls
is very important!).

Good luck.
Brian Rice   rice at dg-rtp.dg.com   (919) 248-6328
DG/UX Product Assurance Engineering
Data General Corp., Research Triangle Park, N.C.
"My other car is an AViiON."