Looking for a big Unix box

[Paul Graham]

In article <1990Feb27.071903.578 at zardoz.cpd.com>, neil at uninet
 (Neil Gorsuch) writes:
'In article <18375 at eerie.acsu.Buffalo.EDU> pjg at autarch.acsu.buffalo.edu
 (Paul Graham) writes:
 [someone is looking to support 100+ users on a box that runs a variety of
  of useful software. neil gorsuch suggests getting a bunch of workstations,
  i suggest getting a larger computer.]
'>last semester we went looking for a timeshare machine.
[actually it was last summer that we decided to get a sun 490 and solbourne
 (sun-4 compatible) series 800 with 2 cpus.  i note that i  don't think
 much of timesharing on a sun 4/60 (sparcstation 1).]
'[neil responds]
'Methinks you don't know how to configure a Sparcstation for maximum
'performance, or that you haven't tried one with > 16 Mbytes yet 8-).
our hardware decisions are somewhat constrained.  we tend not to buy things
from companies like uninet.  not a value judgement just our policy.

'A Sparcstation with 32 Mbytes or more of memory and a couple of 16 mS
'disks is VERY snappy for 10 or so users
hmmmm, mine has 12M and a wren4, which is a 16ms drive i believe, and it's
just this side of not irritating for me.  10 of our emacsing/lisping
users would be a bit much.

[neil suggests cloning the machines with rdist.  we have a lot of per
 cpu licensed software.  a real killer unless you get very sweet deals.]
'(some programs
'need to be re-compiled between the Solbourne and the Sun, and they
'didn't do exactly the same things for systems directory trees).
that's certainly not our experience (except for allegro and ibuki lisp
on the 490).

'Sprinkle the users's files appropriately, with NFS mounts everywhere
'in case a user moves around, and use Yellow Pages to cut down on
'confusion.  Backups are easy with a shared Exabyte drive.
and raise the probabilty that any one dead box will hang the entire
group.  sun does claim that they've put some work into this for 4.1 though.
(i do believe i hate nfs, although not for the same reasons as henry spencer)

'. . . we're
'talking about the best bank for buck for a time sharing machine.
my point exactly.

'As for a 4/490, it's basically a somewhat faster Sparcstation with an
'overpriced card cage.  But for multi-user systems, I would much rather
'have 3 or 4 Sparcstations with 32 or even 64 Mbytes each than a 4/490
'any day, and so would the users that you divvy up per $ spent, if they
'could compare both systems.
i assume that this is simple hyperbole.  you really can't compare a machine
with a large fast cache, 6M/sec disks, adequate support for context switching
and the ability to support a lot of fast memory to a desktop.

'The Solbournes do real well at MIPS/$,
'but their memory is proprietary with no third party availability that
'I know of, while the Sparcstations have dirt cheap memory available
'elsewhere.
most high performance memory systems are somewhat proprietary.  memory per
box is only one cost.

[neil mentions a university that has concluded that workstations are best
 price/performance when you consider disk/ports/memory per $]
'What you have done is choose a Solbourne (a very good
'choice 8-) for part of your "system", but then gone and bought an
'overpriced card cage that is expensive to add memory and disks to for
'the second part.
both are about the same price.  we got the solbourne for price/performance,
we got the sun because it was the best they had and sun will be in business
in 2 years while solbourne (and uninet) may not. the 490 is ``extra''.

[i mumble about how great multi-cpu boxes are]
'I completely agree, but for $'s spent, you probably can't beat the
'desktops, and remember we're talking about a wave of 27 MIPS desktops
'coming out in the next quarter or so.  Not to mention the Taiwan Sparc
'machines due out soon.
right, but i can use the same cpu in my multi box and only buy the memory
and disk once.  or i can get an even better cpu, much better disks and 
more memory available in *one* place and get to reduce my per box costs to
1 or 2.  not to mention the ecl 100 MIPS chips due out in your nearby
compute server soon.

[we spent ~$250 on our two boxes]
'. . . consider these costs, assuming that you have 70 MIPS in your "system":
'6 Sparcstations with 8 Mbytes Sun memory each      $54,000
'6 X 16 Mbytes (4 Mbyte SIMMS)                      $24,000 (approx.)
'6 X 4 Mbytes (1 Mbyte SIMMS)                        $2,500
'6 X 1.3 Gbytes shoeboxes (2 660 Mbyte disks each)  $36,000
'1 Exabyte and 1 1/4" tape drive		     $4,500
'misc costs (software, etc.)                         $5,000
'70 MIPS, 168 Mbytes, 7.9 Gbytes                   $126,000
i'm afraid our software costs would be a bit more than that *per* machine.
we have (or will have shortly) modula, ada, two kinds of lisp, saber-c,
macsyma, mathematica, matlab, lat and various other things that we
pay for.  and we could have bought boxes full of those scsi disks if we
wished and saved a fair bit of money.

'And if I went ahead and spent a hair over $250,000, I would end up
'with a "140 MIPS, 336 Mbytes, 15.8 Gbytes" system.
but *not* in one easy to use package.  our users just connect to a single
place i just manage a single place (sort of).  my current plan is to put
more cpus in the solbourne and turn the 490 into a batch machine assisted
by an i860 attached processor.

[neil talks about being more aggressive and getting things like aviions
 (or ibm powerstations i assume) and having vast quantities of mips which
 is interesting but might not fit the original criteria found on the
 keywords line above.]
mips which i still feel are less usefull spread out across a bunch of small
desktop machines.

'I assume that someone else is providing ethernet based terminal
'servers.
yep.

'The other side of this equation is that someone else spent a
'fair amount for your serial ports, and that with 65 people banging
'away through the ethernet, you waste a lot of ethernet bandwidth and
'host CPU time with the TCP overhead.
nope, not even before we switched to lat.

'Whereas if you split the system
'into smaller chunks with a bunch of serial ports directly in each of
'the Sparcstations you can optimally place the serial lines, you will
'save a noticable amount of $'s, and the hosts will be saving a
'noticable amount of CPU time.  Not to mention that the users won't
'have to log in through an ethernet server and THEN rlogin into one of
'the hosts.
well the serial lines have to get to the users somehow.  we have a three
piece campus with ~20,000 students.  a lot of our access is serial via
a rather expensive data switch, terminal servers are a *major* win.

'>we don't like putting all our eggs in one basket.
'
'But you put it in 2 baskets, where I am proposing putting it in 6
'baskets, which is even more reliable.
i was refering to vendor stability, not hardware stability.

'I'm not saying your choice is
'bad . . ., but I am saying that if you look at things in
'a different light, you perhaps could have spent about half of what you
'spent.  Don't let a Sparcstation with too little memory . . . sour you on the
'possibilities . . .
'And don't get the idea that I am pushing Sparcstations, they just
'happen to be a very cost effective solution right now, plus I know
'that there will be very cheap double digit MIPS versions out from
'Taiwan soon.

well this is entirely too long but . . .
we gave serious thought to a variety of costs, to redundancy, reliability
maintenance, ease of use and efficiency of implementation.  we decided
that unix timesharing on our campus was best done on larger, more centralized
resources.  we think multi-cpu boxes give us the best of both worlds.  we
have a simpler system to administer, we can incrementally increase our
computer power and *all* of our users have access to the large memory, fast
disks and rich software environment without having to figure out which machine
to connect to. (well not entirely but almost).  the biggest plus for a fully
symmetric os is it gives you all these cpus and then arranges that they all
get utilized without user intervention.  i have evidence that the kind of
computing we do here can get done with the equipment we've chosen.  i don't
have any evidence that if we'd spent our money (and i think it's fair to
say that we support the users i was talking about on just the solbourne
since i didn't include peak usage for the 490) on ~$120k worth of
workstations that fact would still hold true. particularly given the fact
that the solbourne appears able to support about 128 users.

there's much to this but it either belongs alt.religion.computers or in
our justification for purchase although it did crop up recently in
comp.arch.