Algorithm needed: reading/writing a large file

Jeffrey W Percival jwp at larry.sal.wisc.edu
Fri Jul 7 04:16:29 AEST 1989


I am writing a C program (Ultrix, VAXstation 2000) to re-arrange a
large disk file.  The file contains a large number of fixed length
records.  My current method is this: read through the file, building 2
arrays in memory:  array1 is an array of some attribute I want to sort
on, and array2 is just array of ascending integers (record numbers in
the first file).  Then I sort array1, with array2 tracking the
movements in array1.  I end up with array2 being a mapping of record
numbers between the input file and the sorted file.  Finally, I loop
through array2 doing a read on file1, record array2[i] and writing to
file2, record i.

Now, I'm not looking for help in the sorting of array1;  my sorting
algorithm is fast and efficient.  What is taking a long time are the
quasi-random seeks in file1 as file2 is sequentially built.

I cannot have all of file1 in memory, but can have more than the one
record I am currently using in this shuffle.  How can I speed this up?
-- 
Jeff Percival (jwp at larry.sal.wisc.edu)



More information about the Comp.unix.wizards mailing list