path hacking tools derived from Joe Buck's stuff

[Edward Vielmetti]

I'm too lazy to polish this up for a real distribution, so I
thought I'd dump it out on alt.sources.  
These are perl sources cobbled from Joe Buck's posting of
several months ago to comp.sources.misc that help in the
process of making usenet maps that correspond to the reality
of how articles are transported around, not the officially
sanctioned view of who is supposed to be connected to who.

The biggest difference between my stuff and Joe's is that
I have split up the single program of his into several
filters that do different things.  Here's a short
summary:
	hiarts		takes history file, emits article names
	artpaths	takes article names, emits Path: lines
	pairs		takes path lines, emits pair and hop info
	plength		takes path lines, emits hop aggregate report

plength is kind of dumb about things, and I would like to go back
and redo at least some of them in something other than perl, since it's
kind of hard to convince people to compile perl just so you can look
at their connectivity.

The usual way I run this is like so:
	hipaths -100 history | pairs | sort +nr
which shows a nice summary of the last hour's traffic.  Here's 
one from now:
    41  4.17 mailrus!ames
    40  4.08 ames!pasteur
    28  4.21 pasteur!ucbvax
    21  4.90 mailrus!tut.cis.ohio-state.edu
    20  4.55 tut.cis.ohio-state.edu!rutgers
    17  4.94 mailrus!ncar
    14  6.29 ncar!tank
    13  6.38 tank!uxc
    13  6.38 uxc!uxc.cso.uiuc.edu
    10  3.50 mailrus!cornell
    10  6.00 uxc.cso.uiuc.edu!m.cs.uiuc.edu
     8  3.38 cornell!batcomputer
     7  3.14 pasteur!cory.Berkeley.EDU
     7  4.57 rutgers!mcnc
     6  4.83 ncar!ames

only the top 15 lines of this one for brevity.

If you have Joe's stuff you'll see that the new feature is a hop
count indication.  This is a metric of the average # of hops that
an article passing through any particular connection takes to 
reach your site.  Long paths around here are 10-12, ordinary
ones are about what you see here.  

Some creative awkery can make this even more interesting.  
Grep all of yesterday's articles out of the history file,
run them through hiarts | artpaths to get the paths, then
take that and run it through "awk -F\! 'NF>15'".  You'll see only
the paths that articles that are generated very far away from you.

Wish list:
	turn this into input for netmap(1)
	get a per-article delay metric
	generate some sort of postscript, netmap or no netmap
	
Watching these snapshots can provide very interesting information about
network connectivity, like noticing that a lot of news goes through
sun!pitstop!sundc!seismo and also through hal!nic.MR.NET!xanth!mcnc.


Anyway, if anyone want to look at some raw data, check the anonymous
FTP area on mailrus.cc.umich.edu for some paths.

--Ed

#	This is a shell archive.
#	Remove everything above and including the cut line.
#	Then run the rest of the file through sh.
-----cut here-----cut here-----cut here-----cut here-----
#!/bin/sh
# shar:	Shell Archiver
#	Run the following text with /bin/sh to create:
#	hiarts
#	artpaths
#	hipaths
#	pairs
#	plength
# This archive created: Sun Oct 30 20:53:28 1988
cat << \SHAR_EOF > hiarts
#!/usr/local/bin/perl

while(<>) {
      ($id, $date, $time, $file) = split (' ');
       next if ($file eq 'cancelled' || $file eq '') ;

      ($ng, $n) = split (/\//, $file);
      $file =~ tr%.%/%;

      print "$file\n";
}
SHAR_EOF
cat << \SHAR_EOF > artpaths
#!/usr/local/bin/perl

die "Can't cd to news spool directory!\n" unless chdir ("/usr/spool/news/news");

$np = $nlocal = 0 ;

while (<>) {

      next unless open (art, $_);

      while (<art>) {
           ($htype, $hvalue) = split (' ');
           if ($htype eq "Path:") {
                print "$hvalue\n" ;
                last ;
           }
      }
}
SHAR_EOF
cat << \SHAR_EOF > hipaths
#!/bin/csh -f
set history=/usr/spool/news/lib/news/$2

tail $1 $history | hiarts | artpaths
SHAR_EOF
cat << \SHAR_EOF > pairs
#!/usr/local/bin/perl

while(<>) {
       @path = split (/!/) ;
       if ($#path < 2) { $nlocal++ ; last;}
       for ($i = 0 ; $i < $#path - 1 ; $i++) {
            $pair = $path[$i] . "!" . $path[$i+1];
            $pcount{$pair} += 1;
            $plength{$pair} += $#path - 1 ;
       }
}


while (($pair, $n) = each (pcount)) {
      printf ("%6d %5.2f %s\n", $n, ($plength{$pair}/$n), $pair);
}

close(sort);
SHAR_EOF
cat << \SHAR_EOF > plength
#!/usr/local/bin/perl

while(<>) {
       @path = split (/!/) ;
       $plength[$#path-1]++ ;
}



for ($i = 1 ; $i < 40 ; $i++) {
       printf ("%2d %6d\n",$i, $plength[$i]);
}
SHAR_EOF
#	End of shell archive
exit 0