Bash, tar, and broken pipe
Byron Rakitzis
byron at archone.tamu.edu
Sat May 18 00:39:26 AEST 1991
In article <heinz.674472878 at cc.univie.ac.at> heinz at cc.univie.ac.at writes:
>In <1991May15.155040.19078 at ssd.kodak.com> weimer at garden.ssd.kodak.com (Gary Weimer (253-7796)) writes:
>>In article <586 at kepler1.kepler.com>, jwu at kepler.com (Jasper Wu) writes:
>>|> I have some problem when using pipelined tar in bash and hope someone
>>|> can help me find out why.
>>|>
>>|> When I do
>>|> zcat foo.tar.Z | tar tvfB -
>>|> or uncompress < foo.tar.Z | tar tvfB -
>>|>
>>|> it gives me the table of contents correctly but reports an error
>>|> message "Broken pipe" to stderr when it finishes. However, if i add
>>|> a "cat" to the pipeline as
>>|> cat foo.tar.Z | uncompress | tar tvfB -
>>|> then it works fine (i.e., no error message). All commands above work fine
>>|> in csh or sh.
>
>> uncompress -c foo.tar.Z | tar tvf -
>> ^^
>
>This does *not* fix the problem.
[...]
>I'm sorry I can't give a solution to the problem - all that I
>know is that bash reports a broken pipe if one of the processes
>making up the pipe is killed or terminated abnormally (well, it
>doesn't have to be terminated abnormally, it only needs to be
>just terminated).
[...]
>It also might be a 'timing prob-
>lem' as zcat surely terminates before tar, and this may cause
>bash to report the pipe as being broken.
Come again? zcat surely terminates before tar? Surely not!
The answer to all this is quite simple. When I do:
cat /usr/dict/words | sed 10q
what happens? sed reads 10 lines of input, and then quits. However, cat
does not know it is writing to a pipe, so it keeps dumping stuff to its
stdout. Something has to stop it, so Unix has a signal called SIGPIPE.
cat therefore dies with the signal "SIGPIPE". Some shells report this
with a "broken pipe" message, because the tail of the pipe died before
the head.
Now, I have not looked at bash source either, but my guess is that
the code does not check the status of each exiting pipeline member,
because in the line
cat foo.tar.Z | uncompress | tar ft -
the "uncompress|tar" section of the pipeline will break when tar finishes
printing the table of contents of the tar file.
Finally, if I type
(echo hi; echo there) | sed 1q
then a shell will most likely not report any error, since "hi\n" and "there\n"
will fit into the pipe buffer.
Hope this clears things up a little. (In my opinion, a shell should never
report broken pipes, since they are usually a part of normal operation.
However, if can be handy to check the exit status of all pipe members.
I have written a shell which does this:
cat /usr/dict/words | tail -r | exit 42
echo $status
prints the output
0 sigpipe 42
but does not report the broken pipe explicitly)
--
Byron Rakitzis
byron at archone.tamu.edu
More information about the Comp.unix.shell
mailing list