How can ethernet traffic be measured?

[Jeffrey Mogul]

In article <288 at asihub.autosys.com> dan at asihub.autosys.com (Dan O'Neill) writes:
>Our site has about 55 hosts on a thin-net based lan.  Due to some
>recent growth, I need a way to analyze the network traffic in order to
>plan for the future.
>
>The network is arranged in a star topology with some DEC ethernet
>Mux's tying the whole thing together.  We have systems running under
>Sun OS, Ultrix and MIPS OS.  We use NFS quite a bit as well as Yellow
>Pages. 
>
>I am not really familar with the traffic problems that occur on a
>local area network and thus am not familiar with 'how much load is too
>much load' type of evaluations.  I've seen Sun's 'traffic' program,
>nice, but what does it all mean?
>
>    What are the watermarks for network performance?

Usually, multi-access networks such as Ethernet are analyzed in
terms of response time as a function of applied load.  In other words,
as the network gets more than lightly loaded, hosts that try to
send packets are blocked for various periods, and this affects the
time it takes to send a packet.  Of course, a network loaded by lots
of hosts will also reduce the amount of bandwidth available to each
of the hosts (provided that the LAN technology is "fair", which is
pretty much true of Ethernet).

It's thus fairly important to know both
	(1) what your applications require in terms of response time
	and bandwidth
	(2) what the peak short-term load averages are on your LAN
before trying to decide if the network is overloaded.

>    How many collisions are too many collisions?

Collisions come from two causes:
	(1) As the load on an Ethernet increases, hosts are increasingly
	likely to try to transmit simultaneously; this kind of collision
	is relatively harmless, since (as shown in a paper by David Boggs,
	Chris Kent, and myself in Proc. SIGCOMM '88) it does not significantly
	decrease the available bandwidth, or increase the average response
	time, until the network is extremely overloaded ... provided that
	you aren't trying to send mostly tiny packets (most applications
	use largish packets).
	(2) If some hardware on the Ethernet is broken, or the network
	is improperly configured, you will get lots "collisions" even when
	the load is low.  These are "bad" collisions; in some sense,
	one is "too many".  Usually, fixing the problem will stop these.
So the trick is to try to figure out which kind of colisions you are getting,
and if they are type #2, fixing the hardware problem.  If they are type #1,
"too many" means your network is overloaded, but that should be obvious
from other indications (like, everything runs slowly).

>    Does UNIX provide any commands for gathering network performance
>    statistics?

BSD-derived systems come with the "netstat" command; "netstat -i" shows
(among other things) collisions per interface (in the column marked
"Coll").  Since this only counts the collisions that happened to
packets sent by this host, you have to compare it to the number of
packets sent ("Opkts") to make any sense of it.  If the collision count
is a factor of ten or more less than that packet count, you're probably
OK (unless you know that your network is lightly loaded, in which case
the collision count should be very low).

Aside from that, I know of no command which is present on all Unix
flavors.  However, many vendors now provide fairly similar things.
The two most useful are load-average displays such as Sun's "traffic"
program) and packet-tracing programs (such as Sun's "etherfind"
program).   I'm pretty sure that you can buy something like "traffic" from
DEC nowadays, but it's not included in the base system and I don't
know how to order it.  Shortly, you'll be able to get the public-domain
program called "tcpdump" for both Sun and DEC machines, and that should
replace "etherfind".

You can also buy complete systems (such as the Sniffer or the SpiderMonitor)
to do this kind of thing, or order PC software (e.g., from FTP Software)
for your IBM-PC (or clones).

Another public-domain program that might help is the "NNstat" package,
sometimes called "statspy" after its primary component.  Soon, this
too will be available for DEC machines (as well as Suns).  NNstat
allows you to gather traffic totals, broken down in all sorts of ways,
but it doesn't tell you how overloaded your LAN is.

Yet another public-domain program, called "nfswatch", might be
helpful in understanding NFS usage on your net. It's already available
for both Sun and DEC Unix systems.

If you've got any VAX/VMS systems, I think DEC sells a package called
EtherNim that does some of this stuff.  DEC also sells a hardware box
called a LAN Traffic Monitor; you need a VMS system to talk to it.

>    Anyone know of any good reference material on the subject?

I gave a paper at this year's SIGCOMM '90 that covers some of this
stuff, from the point of view of someone trying to write the software.
When I wrote that paper, I went looking for formal publications that
talked about network monitoring, and didn't find a whole lot.

Our paper in SIGCOMM '88 might help provide enough context to understand
how heavily loaded an Ethernet can be before it starts causing trouble.

Probably the best source of "how to" material would be the manuals
for systems like the Sniffer, SpiderMonitor, etc.  Maybe, by now,
some tutorials have been given (at shows like InterOp) and the tutorial
notes might be available for purchase.  (There are lots of other
companies selling similar products; I don't want to give the impression
that I'm favoring anyone, I just cannot remember all the names right
now.)

As you may have gathered by now, this area is somewhat in flux; software
is becoming available and people are learning how to manage networks,
but not a whole lot has been written down (especially in handy novice-level
forms).  Within a year or two, I suspect things will be a lot better.

-Jeff