updating an executing setid file when not root

[Doug Gwyn <gwyn>]

?  4.2BSD DOES have the new three-parameter open(2).
You can only set the set-[ug]id bits if the effective ID matches
to file-owner ID, or if the effective ID is 0.  One thing that may
be tripping you up is that any write to the file will clear the
set-[ug]id mode bits (this is another 4.2BSD incompatible feature).
Try chmod(2) after you have finished writing the file.