Multiple 4.0 clients sharing a root partition

[Michael Maciolek]

>Has anyone tried to set up things so that a group of clients whose root
>filesystems reside on the same server can share the same root partition
>under SunOS 4.0? 
>
>...I would much prefer to hear about others' experiences and avoid pitfalls...


A discussion on the use of hard links to save space in the /export/root
partition is discussed in the Sun Software Technical Bulletin, December
issue, p. 2279 (if you have it available).  I think I saw it earlier in
this newsgroup as well.

The main idea is to delete all the big executables in each client's root
directory, replacing them with hard links.  A typical client's root
directory looks like this:

640     vmunix
273     sbin
203     etc
122     var
144     kadb
15      dev
7       home
1       usr
1       tmp
1       sys
1       mnt
1       lib
1       bin

(total 1409 kilobytes)

You can replace vmunix, kadb, and the 5 executables in sbin, with hard
links.  After building the first root directory, which costs about 1400
kb, you can get build additional clients for about 350 kb each.  If you
scrounge around a lot, you might squeeze another 50 or so kb, but I doubt
it's really worth it.

Now - you asked for pitfalls, here's one I've run into - it's very
easy to run out of inodes.  The maximum number of inodes allocated
per cylinder group seems to be 2048.

(Anybody know a way around this?  mkfs won't let me make a smaller
cylinder-group size, and doesn't seem to pay attention to very small
numbers in the "bytes-per-inode" field - it seems you can't get more
than 2048 inodes per cylinder group)

BTW, this is a null issue for most of the smaller disks out there - they
have very small cylinder groups, and thus have plenty of inodes per
megabyte.  My experience is based on the Hitachi DK-815, with a cyl
group size of 8.23 Meg, the CDC Sabre 9720-850 which is a little higher,
and the NEC D2363, with a cyl group size of something over 14 megabytes.

For a cylinder group size of 8.23 Meg per group, 2048 inodes gives you
about one inode per 4 kilobytes.  Trouble is, a client root directory with
500+ files in is will use up inodes at a rate of more than one inode per
kilobyte.  Ten clients only need about 4500 kilobytes (plus a comfortable
margin for shared /tmp and /var/spoo and var/adm spaces), but they'll also
use up about 5000 inodes - you'd need to allocate about 20meg of disk
space to have that many inodes - and you'd have practically no room for
growth!  As soon as someone creates one too many /tmp files,

	/dev/xy0e: out of inodes

It seems inescapable that you must create a huge /export/root partition,
(2-4 times as big as you want it) only to make sure you get enough inodes!

Now, it IS possible to cut back - you probably don't need 96 /dev/winXX
devices (though I have seen a user run out when he had only 64) and you
can cut the pty-tty pseudo-terminal pairs down to about 32 each, but you
still have around 350-400 inodes for every client root directory.

Seems hopless, eh?  But wait!  There is a solution!

The /export/swap partition has EXACTLY the opposite characteristics of
/export/root in terms of kilobyte consumption per inode; a single swap
partition will use many megabytes of disk space, consuming only one inode!
All you need to do is combine the export/root and export/swap files into
one physical disk partition.

As an example, fileserver "sol" has nine clients - "mercury" through "pluto".

Let's say "sol"'s /export/root partition is on /dev/xy0e, and /export/swap
is /dev/xy0f.  Nine clients will require about 4200 kbytes for their
combined root spaces, (that's 1400 for one, plus 8 x 350 for the rest) and
it'd be a nice gesture to leave some more space for client /tmp
directories, /var/adm, /var/spool and so forth.

For swap spaces, you'll have to assess your own needs; If you're really
trimming corners, you might get by with 10 megabytes or less...I've never
tried to see how low you can go with swap space.  So that's 90 meg of swap
for our /export/swap partition.  (I'll ignore filesystem overhead for
now...)

BTW, it's probably a good idea to allocate all your swap spaces first, to
make sure you get big, contiguous chunks of disk.

>From here on, it's easiest for me to just go over the system commands.
I'm doing this off the top of my head - and it's late at night - but I
think I have things pretty close.

Assumptions are - homogeneous sun3 environment, running /bin/csh as root,
and client names are all longer than 1 character.

I'll assume that we've re-partitioned the xy03 and xy0f partitions into a
single combined partition with about 100 meg of usable space (there's
about a 15% total overhead, so about 115 meg raw space, or 230000 sectors)
which is currently empty.

Execute the following commands:

newfs xy0e				# start fresh

mkdir /export/client			# create a mount point
mount /dev/xy0e /export/client		# add this mount to /etc/fstab too!
mkdir /export/client/root
mkdir /export/client/swap

mkfile 10m a b c d e f g h i		# these are just temporary names

cd /usr/etc/install/script		# now we make our clients

set croot = /export/client/root		# client root directories go here
set cswap = /export/client/swap		# client swap files go here
set cexec = /export/exec/sun3

foreach client (mercury venus earth mars jupiter saturn uranus neptune pluto)

	setup_client add $client client 1k $croot $cswap /home $cexec sun3 

	rm $cswap/$client
	mv `ls -1 $cswap/? | tail -1` $cswap/$client	# cutesy, but it works
	if ( $client != "mercury" ) then
		cd $croot/$client
		rm vmunix kadb sbin/*
		ln $croot/mercury/vmunix
		ln $croot/mercury/kadb
		ln $croot/mercury/sbin/sh
		ln $croot/mercury/sbin/init
		ln $croot/mercury/sbin/mount
		ln $croot/mercury/sbin/ifconfig
		ln $croot/mercury/sbin/hostname

		cd /usr/etc/install/script
	endif
end
#
# That ought to do it - you let "setup_client" build each client root and
# swap file, replace the swap file with the one you created earlier (just
# to make sure you get contiguous blocks) and for each client after
# mercury, replace the real vmunix, kadb, and so on, with a hard link to
# the same file in mercury's root directory.
#