Result of request for accounting tools or info

[mem at elrond.UUCP]

Greetings.  A few weeks ago I asked for information about the existence
of useful accounting tools/procedures under Ultrix 1.1 (or, less
specifically, berkeley unix).  I received very few replies, mostly
suggesting that some kernel modifications have to be done in order to
get good accounting data.  Meanwhile, I spent a very small amount of
time looking at the accounting data formats to see what could be done
with it.  the acct(5) man entry describes a data format which seems
quite promising; when the function is enabled, process terminations cause
useful data to be logged in this file (never mind that the manual page
doesn't specify the units of information.  Also that there are a couple
of structure definitions surrounded by an "#ifdef KERNEL" enclosure
which neither seem to relate to anything nor are described).  A little
experimentation showed that the cpu times are stored in units of one
second.  The majority of entries indicate 0 seconds of cpu time
consumed.  So much for interesting information.

One of the responses to my query (from Sid Shapiro, see below) also
mentioned this problem.  Sid went on to say that other aspects of the
accounting system are "broken" under Berkeley 4.2 unix.

The theory of the collection of accounting data via process termination
seems quite good.  It does have some problems, though, including
(at least) these:

	- Memory usage reporting is "broken", as mentioned above.
	- Cpu time granularity of one second is not entirely useful.
	  For instance, today's total CPU times (system and user
	  times combined) for the 8:00-17:00 "prime-time" period
	  come in at under an hour on this Vax 11/780 system with
	  about 16-20 users doing engineering development.  This is
	  clearly wrong.
	- The standard software does not provide a way to get really
	  useful information from the raw accounting data.  Maybe we
	  are unusual, but if nothing else, the minimum we'd like to
	  see is cpu utilization for prime time (as distinct from
	  non-prime-time) periods, in terms of percentage of cpu power
	  actually used.  Many other things could be asked for on
	  top of this, but what sa(8) can give is not among them,
	  unless I am missing something very obvious (but then again,
	  no one else responded to tell me, either).
	- The default action of archiving the accounting data into
	  the savacct file compresses the information and removes
	  the possibility of re-interpreting the data in other ways
	  (so I assume by the textual description; I have yet to find
	  a specification of its format).
	- The format of the acct file could be improved by inserting some
	  self-describing information, even if it were only the record
	  size.  As it is, if the format of the accounting records is
	  changed, so must all programs which use the file be changed.
	  Also, the units of data in the accounting record fields could
	  be specified.


Mark E. Mallett
decvax!savax!elrond!mem		ittatc!sii!elrond!mem


Following are partially condensed responses to my original query:

==========

    We have a program called 'probe' which takes snapshots (probes
kernel statistics) every few minutes and records data in a summary file.
We then produce a weekly report showing watermarks (greatest # of procs,
login sessions, etc.)  and averages (% cpu util, etc.) for each day,
week, etc.

    We also ported the ATT Sys3 accounting package to 4.1 and 4.2
(nontrivial), which gives us the type of summaries and breakdown you
require.  It requires a SysIII source license and would probably require
a lot of legal hassle to license from us.

    There is a commercial package from Aim Technology (in CA) that will
give you the type of reporting you require (it was recently announced).
I believe it is several thousand $.

			H. Morrow Long
			Member Research Staff - Knowledge Based Systems
			ITT-ATC Advanced Technology Group
			1 Research Drive Shelton, CT  06484
			Phone #: (203)-929-7341 x. 634

path = {allegra bunker dcdvaxb dcdwest ucbvax!decvax milford mit-eddie
	psuvax1 qumix sii supai tmmnet yale}!ittatc!long

==========

We rolled our own (for 4.1 and 4.2 BSD). It charges different people
different rates and recognizes peak and non-peak timer. Lots of kernel
hacks were involved though.
---
Bill Sebok			Princeton University, Astrophysics
{allegra,akgua,cbosgd,decvax,ihnp4,noao,philabs,princeton,vax135}!astrovax!wls

==========

I looked into accounting stuff a while back, and gave it up.  Beware
that the 4.2 (and ultrix 1.0 and 1.1) accounting is broken.  DEC
claims that 1.2 fixes the problems.  The bugs are in the accounting
file (actually the parts of the kernel that dump info to the
accounting file) - so fixing the accounting file readers does not fix
the problem.

The problems  show up as negative memory values from an "sa" report.

It appears that the memory integrals are not being properly 
normalized when the accounting info is calculated.  

There's another (undocumented) change in 4.2 logs: the CPU system and
user times are now logged in units of seconds. Before, they were in
units of sixtieths of a second.  The change menas that SA output is off
by a factor of 60 (SA code appears not to have changed from 4.1 to 4.2).
Now, with SA, the -j option yields minutes (not second), and the CPU
times (without -j option) are now in hours.

Also, SA's memory usage is too high by a factor of 50 (it still assumes
that pages are 512 bytes).

-- 
Sid Shapiro -- Wang Institute of Graduate Studies
	ss at WANGINST.EDU		(ss%wanginst.CSNET at Csnet-Relay.ARPA)
	(617)649-9731		decvax!wanginst!ss


==========

Mark E. Mallett
decvax!savax!elrond!mem		ittatc!sii!elrond!mem