On the silliness of close() giving EDQUOT

[Dan Bernstein]

Skip to ``function'' if you want to see a couple of nasty challenges
rather than all the details.

In article <1990Oct19.055913.7103 at athena.mit.edu> jik at athena.mit.edu (Jonathan I. Kamens) writes:
>   Dan, I think this is one of those issues where we are arguing because you
> are convinced that the way you think Unix should be is the One True Way and
> that everyone who thinks differently from you is wrong.

One of the problems with your writing style is that you put ``I think''
before each of your opinions. One of the problems with your reading
style is that you assume everyone else shares your stilted view of
presentation.

I don't pretend that what I say is gospel. *Obviously* it's only what I
believe. Stop making me out to be some sort of religious fanatic, and
start presenting rational arguments that might convince me of something.

> Or, at least, that's
> the tone you project in your postings on this topic (you projected exactly the
> same tone when we discussed syslog).

Oh? Here's the model for your syslog argument: ``1. Dan says a logging
system shouldn't be *forced* under central control. 2. Dan says that
central control is a reasonable *default*. 3. syslog has central control
as its *only* option. 4. Therefore syslog is sufficient for Dan.''

That's simply ridiculous. I've italicized above the words that you kept
ignoring in that discussion. I can configure stderr, even though central
control is a good *default*. I can use stderr securely, even though
insecure /dev/log may be a good *default*. I can use stderr reliably,
even though unreliable syslog may be a good *default*.

Your final article went through each of the above points and many
others. You kept repeating ``well, you say that central control and
/dev/log are enough, so why don't you like syslog?''

Now *that's* a ``One True Way'' attitude.

Someday I hope you will realize how restrictive syslog is. I was never
really angry at it until I tested a new (slightly buggy, of course)
version of telnetd. Was I supposed to disable all the syslog()s while
testing? Eventually I had to do just that, so that I wouldn't flood the
real system logs with messages that could not be distinguished from
important messages from the running telnetd. I hope you hit a similar
wall sometime.

In the meantime, don't accuse *me* of ``projecting'' a ``tone.'' It was
your refusal to respond rationally that told me you weren't paying
attention.

>   Actually, my close(2) says nothing about EINTR.  Does that mean my program
> doesn't have to be prepared to deal with an interrupted close()?

I talk about this in another article.

> |> No other close() errors make sense.
>   It is your *opinion* that no other close() errors make sense.  There are a
> whole lot of people, including the designers of NFS and the designers of AFS,
> who may not agree with you.

This is exactly what I hate the most about your argument style. I
express opinion X. You can't figure out a rational response to X. So you
say (substitute appropriately): ``It is your *opinion* that X. You are
extremely opinionated. There are lots of people who disagree with you. I
disagree with you.''

I say that EINTR and EBADF make sense, given the documented function of
close(). You say they don't. I challenge you to explain why EDQUOT is a
sensible error for the ``delete a descriptor'' function to return.

Skip to ``Suppose'' for a second challenge.

>   Actually, I take that back.  I have heard that the designers of AFS are
> working on a fix for a later release which will report cause quota problems to
> be reported to users in a more timely manner, although I don't know what
> mechanism they are using to do so.

Excellent. It's good to hear that filesystem designers are joining the
real world.

>   However, that doesn't change my main point, which is that I do not think it
> is reasonable to place a restriction on all future filesystems developed for
> Unix that they all must detect quota problems immediately.

Did I ever claim such a restriction? No. In fact, I'm strongly against
any rules that restrict future extensions. However, it is most logical
for EDQUOT to be detected upon the offending write()---and current
filesystems *can do so* without undue implementation difficulty. So it's
silly for close() to return EDQUOT.

>   The only justification you have given for your claim that no other close()
> errors "make sense" is that you don't think filesystems should work that way. 
> I'm afraid that's just not good enough.

Here goes your argument style again. Jon, you are the one arguing for a
change from historical implementations, so it's incumbent upon you to
prove your point. You say that EDQUOT ``makes sense'' for close(), but
the only justification you've given is that NFS does it that way. Surely
if you feel so strongly about this issue then you have some rational
reason for your belief? How about sharing this reason with the rest of
us?

> |> You say that close() should be able to return -1/EDQUOT. That's hogwash.
> |> EDQUOT can and should be detected immediately upon the write() that
> |> triggers it. There's no reason that the system call for saying ``Okay,
> |> forget about this descriptor, I'm done with it'' should produce an errno
> |> saying ``But wait! I neglected to mention this to you when you actually
> |> wrote the data, but you're out of space! Don't you dare forget about
> |> this descriptor when you're out of space! Hold the presses!''
>   You are still back in the days when all the world was UFS.  Close() doesn't
> mean what you say above anymore (actually, it's possible to argue that it
> never did).  What it means is, "I'm done with this file descriptor, so please
> do any finishing touches that you need to on it and then close it up, and let
> me know if you have any problems with that."

You're fantasizing. You say close() means something entirely different
from what Bach, and my man pages, and lots of other references say.

> |> Perhaps you don't yet see how silly that is. Has it occurred to you that
> |> the application may have erased the data that it wrote to disk? Are you
> |> going to insist that every write() be backed up by temporary buffers
> |> that accumulate a copy of all data written until the program dies? Well?
>   First of all, the problem isn't as large as you are trying to make it out to
> be, because most of the programs I've worked/used don't even deal well with a
> write() failing because of a quota problem; they just say, "Oops, I can't
> write, I'm going to give up completely,"

I agree with your pragmatic point, but we're talking about reliability
here.

>   For the few programs that are planning on doing serious error recovery if
> they can't write to the disk and that are planning on trying to preserve all
> data even if a disk write fails, then yes, I expect them to do some sort of
> buffering.  If you want a robust program, then you put more work into it.

Sorry, Jon, this fails. Has it occurred to you that several processes
may write to a file before one of them does the close() that writes it
out to disk? How the hell do you expect data to be buffered in the
meantime? Your ``robust program'' is another fantasy.

An I/O error makes reliable disk operations very difficult. At some
level there has to be enough replication to make failures invisible.
Unless the OS does this, each program has to write data to enough places
that an I/O error can be handled. If it does so, EIO is not a disaster.

A quota error, however, *must* be reported as soon as possible. If
EDQUOT is saved up until some system buffer is flushed, a program may
have all its work destroyed, even if it checks your precious close()
error. After all, if some other program happens to be accessing the same
file, each write() and close() could succeed. Do you understand the
problem here?

     [ Jon makes an entirely unsupported claim, and I satirize his ]
     [ lack of justification: ]
> |> I fail to see your logic. Can I substitute any two system calls there?
> |> ``Just as it is reasonable for unlink() to return an error when the file
> |> doesn't exist, it is reasonable for setuid() to do so, and the man page
> |> should be updated to reflect that.''
> |> Now what is your argument?
  [ Jon criticizes my analogy, failing to realize that he's implicitly ]
  [ condemning his own lack of logic ]
> Your introduction of fallacious reductio ad absurdum arguments into
> the discussion just makes it harder to discuss the subject matter with you
> intelligently.

Read carefully and think about what you just wrote.

> |> No. I am saying that there is no excuse for a filesystem to let a
> |> program write() more than the user's quota without giving an error.
>   This is your opinion, and you are entitled to it.  You are also entitled to
> write code that does not check close() for EDQUOT, in which case your code
> will lose data when writing to remote filesystems and your customers will
> complain to you, and you can tell them that you can't fix it because close()
> *shouldn't* return EDQUOT.

Okay, wiseass. My pty program writes output on the pseudo-tty to its own
output, the normal stdout passed as usual from pty's invoker.

Now let's say I'm trying to obey your ``reliability'' constraints.
Rather than letting the system close the descriptor upon exit(), I take
care to close() it myself. I check the return value and error code.

Suppose it now returns an error. Uh-oh, EDQUOT. How do I handle
it? I'm following your prescriptions, and I've saved everything I've
written in an internal buffer. But I have no idea what other programs
might have written to the same descriptor. They won't have found out
about the EDQUOT, even though they checked close()---because the system
buffer hasn't yet been flushed. How am I supposed to handle this error?

Jon, I don't think you've thought out the consequences of this problem.
I don't think you've even considered the above situation. I challenge
you to describe a robust EDQUOT handler.

Feel free to apologize through e-mail rather than in public.

> And I'll admit that i may not be able to see forever into the future and
> tell that we will never ever be able to justify a filesystem that doesn't
> detect quota problems on write().

I agree with your general point; this is one of the many arguments I've
put forth in comp.std.unix for why files shouldn't be forced into the
filesystem abstraction. However, I'm not talking about future
restrictions. I'm talking about what we have *now*. The only example of
real-world EDQUOT close() behavior (that won't be fixed soon) is NFS,
which everyone agrees is a botch. You may be right, and timely EDQUOT
reporting may be difficult on some future file type. But it isn't now.

> |> ``But then it has to send a request immediately over the network and
> |> wait to find out how much space is left!'' you say. Not so. Does TCP
> |> force each side to stay synchronized on every packet? Of course not. The
> |> file server can just pass out ``allocations'' of disk space. Things only
> |> slow down when you're very close to the hard limit.
>   Please explain what you're proposing here -- I don't quite get it.

Windows. Just like TCP windows. This idea, like mostly everything else
in computer ``science,'' has been around since at least the sixties. You
make the most common operations faster by keeping the information they
need readily available.

---Dan