Read only Source trees?

[Chris Torek]

In article <BGLENDEN.90Sep11155146 at mandrill.cv.nrao.edu>
bglenden at mandrill.cv.nrao.edu (Brian Glendenning) writes:
>Can anyone tell me what the usual tricks are when dealing with
>readonly source hierarchies (i.e. we have 5 different binary formats
>and we're tired of maintaining several versions of identical source
>trees (the sources are changing somewhat rapidly and the extra copies
>take not insignificant amounts of disk)). I'm not clear what the best,
>or even usual, solution to this (presumably) common problem is.

One solution, operating now at UC Berkeley and almost working here
at UMCP-CSD (there are still some programs that must be cleaned up):

 1. Obtain pmake (Adam de Boor's `parallel make') as modified for
    4.3BSD-reno (these mods may be optional; I have not looked at
    the unmodified version).

 2. Put symlinks in your kernel, or a hack into pmake for `pretend
    symlinks' of some sort (see step 4 below).

 3. Change all your makefiles.  This is the hard part.  In the process,
    you generally want to move each program's source into a separate
    subdirectory, and move the source version of the manual for that
    program to the same directory.  In some cases this requires thought
    and/or use of special pmake features.  In most cases, however, you
    wind up with a situation like one of these:

    a) program /usr/bin/foo is built from one C source file.  The
       Makefile in /usr/src/bin/foo reads, in its entirety:

	PROG=	foo
	.include <bsd.prog.mk>

       (you can of course add comments and blank lines if you like).

    b) program /usr/bin/foo is built from several C source files.  The
       Makefile in /usr/src/bin/foo reads:

	PROG=	foo
	SRCS=	foo.c bar.c baz.c
	.include <bsd.prog.mk>

    c) program /usr/sbin/foo is built from one C source file, but its
       manual goes in section 8:

	PROG=	foo
	MAN8=	foo.0
	.include <bsd.prog.mk>

    d) program foo has no manpage (e.g., the C preprocessor): add the
       line

	NOMAN=	noman

       (or `NOMAN=an_island' if you prefer :-) ).  <bsd.prog.mk> simply
       keys off the existence of this make-variable and avoids building
       and installing a man page.

    e) directory makefiles: /usr/src/bin/Makefile reads something like

	SUBDIR=	cat cp csh ed ls sh stty test
	.include <bsd.subdir.mk>

    f) library makefiles: these look something like

	LIB=	foo
	SRCS=	foofile.c fooio.c foosubr.c
	MAN3=	fooopen.0 fooread.0 foocompute.0
	MLINKS=	fooopen.0 fooclose.0 \
		fooread.0 foowrite.0
	.include <bsd.lib.mk>

    g) complicated makefiles: you will need to use ${.CURDIR} to give
       full path names for source files, because of step 4 below.  Read
       the pmake documentation for details.

    Machine-specific source files (such as the VAX `arff' console-media
    manipulation program) go in subdirectories as usual, and the parent
    directory's makefile gets a bit uglier:

	.if   ${MACHINE} == "sun4"
	SUBDIR+=installboot
	.elif ${MACHINE} == "tahoe"
	SUBDIR+=dlmpcc enpload
	.elif ${MACHINE} == "vax"
	SUBDIR+=arff rxformat
	.endif

    When you have completed this step, you can simply

	cd /usr/src; make depend; make; make install; make clean

    and everything will be recompiled and reinstalled.  (This takes
    a few hours or so....)

 4. This is the magic part.  All of the above is simply preliminary
    work.  Mount /usr/src read-only.  Make /usr/obj a symlink if
    necessary, or simply mount something on /usr/obj.

	[on the machine on which /usr/src is to be kept, where it
	 is mounted read/write:]
	$ cd /usr/src
	$ make cleandir		# remove any leftover stuff

	[on every machine that it to use /usr/src]
	$ mkdir /usr/obj; find /usr/src/* -type d -print |
	> sed 's,/usr/src/\(.*\),mkdir /usr/obj/\1\
	> ln -s /usr/obj/\1 /usr/src/\1/obj,' | sh

    This takes a while.  When it is done, run

	$ cd /usr/src; make depend; make; make install

    on all the machines sharing /usr/src.

Note that 4.3BSD-reno comes with a new manual page scheme, but it is
not necessary to buy into this.  A small modification to <bsd.man.mk>
allows using the old method without changing *any* makefiles.  (By
not sharing /usr/share/mk, or by putting ${MACHINE} tests in these
files, you can even split between new and old on different architectures.)

Although step 3 above is painful, it is worth it: it separates the
*specifications* for the sources for each program (which differ for
each program) from the *semantics* of how to make each program (which
are almost always identical; the few exceptions get bigger makefiles).
If the semantics change---e.g., if dependencies are to be maintained
by some other method than `mkdep'---virtually no changes to /usr/src
are necessary.  Only the `master' /usr/share/mk/*.mk files (and their
sources in /usr/src/share/mk) are affected, and, occasionally, those
exceptional makefiles.

Want to install with default owner `root' instead of `bin'?  Just
change <bsd.own.mk> and <bsd.lib.mk>.  Want to build libraries with
extra debugging information?  Just change <bsd.lib.mk>.  And so on....

This scheme may not be the best, but it works.  It solves two
problems:  shrinking the makefiles (moving common semantics to a
commmon place), and sharing the sources (one source tree suffices for
any number of architectures).  If you do cross-compilation you can even
handle that, if a bit grotesquely, by making /usr/obj itself a symlink
and pointing it to different object trees.  (What is really needed here
is Plan-9 style per-process mounts.)
-- 
In-Real-Life: Chris Torek, Univ of MD Comp Sci Dept (+1 301 405 2750)
Domain:	chris at cs.umd.edu	Path:	uunet!mimsy!chris