Smail3, SunOS and shared libraries

[Ronald S. Karr]

The Smail3 program does not run well on several recent releases of
SunOS, and I don't understand why.  I do not actually have a Sun
myself, so I can't look into the problem directly, though some of the
sites that have Smail3 sources and Sun machines have tried to find
the problem, and so far nobody has reported a solution.

The problem is this:  Smail3 cannot exec programs that require
shared libraries, if the smail binary is setuid to root, or is run
by root.  One time, over the phone, I talked someone through the
process of getting smail to exec an interactive shell by mailing
a message through smail.  This worked, but only commands without
shared libraries could be executed.

I can only think of a few things in the smail sources that might
cause these problems, and so everyone that I have talked to about
this believes that these should cause a problem.  These are the
things that may (even remotely conceivably) be related:

	1.  the environment is completely reinitialized.

	2.  all file descriptors greater than 2 are closed, both
	    at startup and before execing something in a forked
	    process.

	3.  setgid() and setuid() are used to set the effective
	    (and hopefully the real) user and group IDs before
	    execing another program.

I would very much appreciate a response from anyone who has an idea
on what the problem is and how it can be fixed.

One last note:  Smail3 runs just fine under SVR4.0, which has a very
similar (though not identical) shared library structure.
-- 
	tron |-<=>-|		ARPAnet:  veritas!tron at apple.com
      tron at veritas.com		UUCPnet:  {amdahl,apple,pyramid}!veritas!tron