reap(8): expire news as space needed

[Arnd Merlevede]

dt at yenta.alb.nm.us (David B. Thomas) writes:

hallo andreas, vielleicht interessiert Dich
die follgende Msg falls Dus nicht selbst gelesen hast.

Uebrigns, liegt noch die Source von nn irgendwo ?
Und ist des schwer die zu installieren ? Ich wuerd sie gerne 
im ET-CipPool installieren (ala faui09)
?
Bis denn Arnd


>This message contains the README, followed by the source:

>README for reap version 1.1 by David B. Thomas (dt at yenta.alb.nm.us).

>A Problem:
>	News volume fluctuates wildly...sites with small disks must either
>expire aggresively, often deleting more than necessary, or worry that an
>unexpected huge dose of news might overfill the disk.


>A Solution:
>	Implement a tool that expires according to a user-defined scheme
>until sufficient freespace is reclaimed, then stops, leaving as much
>juicy news online as is feasible.  Reap does this.


>Expire Does Two Jobs:
>	Both Bnews and Cnews expires really do two jobs:
>		1. trim history files
>		2. delete outdated articles
>Thanks to some inspired jootsing (acronym for "jumping out of the system")
>by Mike Murphy (mrm at sceard.com) and others, it is more than possible to
>separate those two functions.  This is, of course, in keeping with the
>unix philosophy of one tool doing one job well!


>What Reap Does:
>	Reap only takes care of the second job: deleting old articles.
>It works by checking freespace, and processing one line at a time from a
>list of expire functions, until the desired freespace is attained.
>Each expire function consists of an age limit in days (decimals okay)
>and a list of newsgroups to process or not process, sys file style.  Ex:

>	.5	alt.sex.pictures,talk,!talk.bizarre,junk
>	1	rec,!rec.games,!rec.humor

>This example would check freespace, and if more space is needed, expire
>to .5 days everything in alt.sex.pictures, talk (except for talk.bizarre)
>and junk.  Then it would stop and check freespace again.  If still more
>space is needed, it would expire to 1 day everything in rec except
>rec.games.* and rec.humor.*.  It's that simple.


>Okay...So How Do I Arrange To Trim The History Files, Since Reap Can't:
>	Included in this distribution is a shell script (mostly written
>by Mike Murphy) to handle Cnews history files.  It shouldn't be too
>difficult to do something similar for Bnews, or you can give in and use
>the original expire utility with options that tell it to expire the
>history files only...but I think this will be slow.  It just comes down
>to removing lines from ordinary text files, based on their contents.
>Murphy and I used awk.


>But Is It Fast:
>	Yes, largely because it doesn't have to do much.  Even "find | rm"
>is slower because find is repeatedly exec-ing rm.  "rm -rf" has me beat,
>though, I'll bet!  :-)

>	Since the functions of deleting articles and trimming history
>are separate, I now run reap every six hours, but trim the history list
>just once a day.  That effectively keeps my disk space up to snuff, but
>only thrashes at the history file in the middle of the night.


>Credits:
>	I owe a lot to Mike Murphy for inspiring me with his "trasher"
>system.  I also owe a lot to all of your netters who will flood me with
>more suggestions and improvements in the coming weeks (hint, hint!).

>						little david
>						dt at yenta.alb.nm.us

>ps - reap is freeware.  No strings(1) attached.


>#!/bin/sh
># This is a shell archive (shar 3.44)
># made 03/28/1991 02:51 UTC by dt at yenta
># Source directory /u/dt/src/reap
>#
># existing files will NOT be overwritten unless -c is specified
>#
># This shar contains:
># length  mode       name
># ------ ---------- ------------------------------------------
>#    934 -rw-rw-r-- Install
>#    339 -rw-rw-r-- MANIFEST
>#    900 -rw-rw-r-- Makefile
>#   2918 -rw-rw-r-- README
>#    969 -rwxrwxr-x exphist
>#   3567 -rw-rw-r-- lib.c
>#   1611 -rw-rw-r-- list.sample
>#   2497 -rw-rw-r-- main.c
>#   6129 -rw-r--r-- reap.8
>#   2845 -rw-rw-r-- reap.c
>#   1195 -rw-rw-r-- reap.h
>#
># ============= Install ==============
>if test -f 'Install' -a X"$1" != X"-c"; then
>	echo 'x - skipping Install (File already exists)'
>else
>echo 'x - extracting Install (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'Install' &&
>X
>To install:
>X
>1. Edit tops of Makefile and reap.h to fit your system.
>X   [If you are installing this on a non-system-V machine, you will need
>X   to rewrite the freeblox() function at the top of lib.c.  This should
>X   be very easy.  Please email your working mods to me.]
>2. make
>3. Compose a sample function list file or use the sample provided to
>X	test reap (use the -n option to avoid really removing files).
>4. su and make install.
>5. Arrange to trim your news system's history list regularly.  If you have
>X	Cnews, try the "exphist" script provided, run from cron.  That
>X	script requires that a tiny file, /usr/lib/news/histdays, containing
>X	the number of days of history data to keep, or else you can hardcode
>X	in your favorite number.  If you have Bnews, I don't have a nifty
>X	suggestion, other than using expire(8) with a ridiculously long
>X	article expire time, but a sane history expire time.
>6. Arrange to run reap from cron.
>SHAR_EOF
>chmod 0664 Install ||
>echo 'restore of Install failed'
>Wc_c="`wc -c < 'Install'`"
>test 934 -eq "$Wc_c" ||
>	echo 'Install: original size 934, current size' "$Wc_c"
>fi
># ============= MANIFEST ==============
>if test -f 'MANIFEST' -a X"$1" != X"-c"; then
>	echo 'x - skipping MANIFEST (File already exists)'
>else
>echo 'x - extracting MANIFEST (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'MANIFEST' &&
>Included with this distribution are:
>X
>README		- general description and credits
>Install		- hints on how to install reap and get it going
>Xexphist		- history trimmer utility for Cnews
>list.sample	- sample function script file for reap
>reap.8		- man page
>lib.c		- source
>main.c		- source
>reap.c		- source
>reap.h		- source
>Makefile	- makefile
>SHAR_EOF
>chmod 0664 MANIFEST ||
>echo 'restore of MANIFEST failed'
>Wc_c="`wc -c < 'MANIFEST'`"
>test 339 -eq "$Wc_c" ||
>	echo 'MANIFEST: original size 339, current size' "$Wc_c"
>fi
># ============= Makefile ==============
>if test -f 'Makefile' -a X"$1" != X"-c"; then
>	echo 'x - skipping Makefile (File already exists)'
>else
>echo 'x - extracting Makefile (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'Makefile' &&
>#----------------------------
># parameters for installation
>#----------------------------
>X
># directories where manual and executable are to be installed
>BINDIR = /usr/lib/newsbin/expire
>MANDIR = /usr/man/man8
>X
># owner, group and file mode for manual and executable
>EXEOWNER = bin
>EXEGROUP = bin
>EXEMODE = 775
>MANOWNER = bin
>MANGROUP = bin
>MANMODE = 664
>X
>X
># your favorite C compiler
>#CC = cc
>CC = shcc
>X
># directory access functions library, if not in standard C library.
>LIB = -ldirent
>X
>X
>#-------------
># boring stuff
>#-------------
>X
>OBJ = main.o lib.o reap.o
>X
>reap: $(OBJ)
>X	$(CC) -s -o reap $(SHLIB) $(OBJ) $(LIB)
>X
>$(OBJ): reap.h
>X
>install: reap reap.8
>X	cp reap $(BINDIR)
>X	chown $(EXEOWNER) $(BINDIR)/reap
>X	chgrp $(EXEGROUP) $(BINDIR)/reap
>X	chmod $(EXEMODE) $(BINDIR)/reap
>X	cp reap.8 $(MANDIR)
>X	chmod $(MANMODE) $(MANDIR)/reap.8
>X	chown $(MANOWNER) $(MANDIR)/reap.8
>X	chgrp $(MANGROUP) $(MANDIR)/reap.8
>SHAR_EOF
>chmod 0664 Makefile ||
>echo 'restore of Makefile failed'
>Wc_c="`wc -c < 'Makefile'`"
>test 900 -eq "$Wc_c" ||
>	echo 'Makefile: original size 900, current size' "$Wc_c"
>fi
># ============= README ==============
>if test -f 'README' -a X"$1" != X"-c"; then
>	echo 'x - skipping README (File already exists)'
>else
>echo 'x - extracting README (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'README' &&
>X
>README for reap version 1.1 by David B. Thomas (dt at yenta.alb.nm.us).
>X
>A Problem:
>X	News volume fluctuates wildly...sites with small disks must either
>Xexpire aggresively, often deleting more than necessary, or worry that an
>unexpected huge dose of news might overfill the disk.
>X
>X
>A Solution:
>X	Implement a tool that expires according to a user-defined scheme
>until sufficient freespace is reclaimed, then stops, leaving as much
>juicy news online as is feasible.  Reap does this.
>X
>X
>Expire Does Two Jobs:
>X	Both Bnews and Cnews expires really do two jobs:
>X		1. trim history files
>X		2. delete outdated articles
>Thanks to some inspired jootsing (acronym for "jumping out of the system")
>by Mike Murphy (mrm at sceard.com) and others, it is more than possible to
>separate those two functions.  This is, of course, in keeping with the
>unix philosophy of one tool doing one job well!
>X
>X
>What Reap Does:
>X	Reap only takes care of the second job: deleting old articles.
>It works by checking freespace, and processing one line at a time from a
>list of expire functions, until the desired freespace is attained.
>Each expire function consists of an age limit in days (decimals okay)
>and a list of newsgroups to process or not process, sys file style.  Ex:
>X
>X	.5	alt.sex.pictures,talk,!talk.bizarre,junk
>X	1	rec,!rec.games,!rec.humor
>X
>This example would check freespace, and if more space is needed, expire
>to .5 days everything in alt.sex.pictures, talk (except for talk.bizarre)
>and junk.  Then it would stop and check freespace again.  If still more
>space is needed, it would expire to 1 day everything in rec except
>rec.games.* and rec.humor.*.  It's that simple.
>X
>X
>Okay...So How Do I Arrange To Trim The History Files, Since Reap Can't:
>X	Included in this distribution is a shell script (mostly written
>by Mike Murphy) to handle Cnews history files.  It shouldn't be too
>difficult to do something similar for Bnews, or you can give in and use
>the original expire utility with options that tell it to expire the
>history files only...but I think this will be slow.  It just comes down
>to removing lines from ordinary text files, based on their contents.
>Murphy and I used awk.
>X
>X
>But Is It Fast:
>X	Yes, largely because it doesn't have to do much.  Even "find | rm"
>is slower because find is repeatedly exec-ing rm.  "rm -rf" has me beat,
>though, I'll bet!  :-)
>X
>X	Since the functions of deleting articles and trimming history
>are separate, I now run reap every six hours, but trim the history list
>just once a day.  That effectively keeps my disk space up to snuff, but
>only thrashes at the history file in the middle of the night.
>X
>X
>Credits:
>X	I owe a lot to Mike Murphy for inspiring me with his "trasher"
>system.  I also owe a lot to all of your netters who will flood me with
>more suggestions and improvements in the coming weeks (hint, hint!).
>X
>X						little david
>X						dt at yenta.alb.nm.us
>X
>ps - reap is freeware.  No strings(1) attached.
>SHAR_EOF
>chmod 0664 README ||
>echo 'restore of README failed'
>Wc_c="`wc -c < 'README'`"
>test 2918 -eq "$Wc_c" ||
>	echo 'README: original size 2918, current size' "$Wc_c"
>fi
># ============= exphist ==============
>if test -f 'exphist' -a X"$1" != X"-c"; then
>	echo 'x - skipping exphist (File already exists)'
>else
>echo 'x - extracting exphist (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'exphist' &&
>#! /bin/sh
># exphist - expire history file
>X
># =()<. ${NEWSCONFIG-@<NEWSCONFIG>@}>()=
>. ${NEWSCONFIG-/usr/lib/news/bin/config}
>X
>PATH=$NEWSCTL/bin:$NEWSBIN/expire:$NEWSBIN:$NEWSPATH ; export PATH
>umask $NEWSUMASK
>X
>days=`cat /usr/lib/news/histdays`
>X
>lock="$NEWSCTL/LOCKexpire"
>ltemp="$NEWSCTL/L.$$"
>Xecho $$ >$ltemp
>trap "rm -f $ltemp ; exit 0" 0 1 2 15
>if newslock $ltemp $lock
>then
>X	trap "rm -f $ltemp $lock ; exit 0" 0 1 2 15
>Xelse
>X	echo "$0: expire apparently already running" | mail "$NEWSMASTER"
>X	exit 1
>fi
>X
>cd $NEWSCTL
>rm -f history.n history.n.pag history.n.dir
>now=`getdate now`
>age=`expr 86400 \* $days`
>ago=`expr $now - $age`
>awk "{split(\$2,dates,\"~\");if(dates[1]>$ago)print \$0}" history >history.n
>mkdbm history.n
>[ -s history.n ] &&
>mv history history.o &&    # install new ASCII history file
>mv history.n history &&
>rm -f history.pag &&       # and related dbm files
>rm -f history.dir &&
>mv history.n.pag history.pag &&
>mv history.n.dir history.dir
>Xexit 0
>SHAR_EOF
>chmod 0775 exphist ||
>echo 'restore of exphist failed'
>Wc_c="`wc -c < 'exphist'`"
>test 969 -eq "$Wc_c" ||
>	echo 'exphist: original size 969, current size' "$Wc_c"
>fi
># ============= lib.c ==============
>if test -f 'lib.c' -a X"$1" != X"-c"; then
>	echo 'x - skipping lib.c (File already exists)'
>else
>echo 'x - extracting lib.c (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'lib.c' &&
>X
>/*
>X * lib.c
>X *	subroutines needed by reap utility
>X */
>X
>#include "reap.h"
>X
>/*
>X * freeblox(devno)
>X *
>X * THIS WILL REQUIRE MODIFICATION TO WORK UNDER NON-SYSTEM-V UNIX.
>X * IT SHOULD BE EASY.  PLEASE EMAIL WORKING MODS TO THE AUTHOR:
>X *			dt at yenta.alb.nm.us
>X *
>X *	assuming devno is the device number of a file system,
>X *	this function returns the free space on that file system,
>X *	in blocks (whatever that means for that filesystem) as an int.
>X *
>X */
>X
>freeblox(devno)
>int	devno;
>{
>X	static struct ustat	ust;
>X
>X	if (ustat(devno, &ust))
>X		ouch ("%s: ustat() failed\n");
>X
>X	return (ust.f_tfree);
>X
>} /* freeblox() */
>X
>X
>X
>X
>/*
>X * parse(cmd)
>X *	execute a line from the script file
>X *
>X * valid command lines:
>X *	a blank line is ignored
>X *	# comment (ignored)
>X *	days filespecs
>X *
>X * filespecs is a comma separated list of any combination of the following:
>X * 1. a directory (newsgroup) to include, without recursion.
>X *	ex:  alt.sources.
>X * 2. a directory to include, recursively.
>X *	ex:  alt.sources
>X * 3. a directory to exclude, without recursion.
>X *	ex:  !alt.sex.
>X * 4. a directory to exclude, recursively.
>X *	ex:  !alt.sex
>X */
>X
>parse(cmd)
>char	*cmd;
>{
>X	register int	l;
>X	int		recurseflag;
>X	double		days;
>X	char		*t,
>X			*p,
>X			*q,
>X			*seps = ",\n ";
>X
>X
>/* skip initial whitespace */
>X	p = cmd;
>X	while (isspace(*p))
>X		++p;
>X
>/* ignore blank lines or comments */
>X	if (*p == '\0' || *p == '#')
>X		return(1);
>X
>/* read the expire time from the line as a floating point number,
>X * then figure out what the timestamp on a file that old would be */
>X	age = now - (time_t) (atof(p) * SECINDAY);
>X
>/* skip ahead to the filespec list */
>X	while (!isspace(*p))
>X		++p;
>X	while (isspace(*p))
>X		++p;
>X
>/* initialize exclusion list to null list */
>X	preclude();
>X
>/* for each filespec in list */
>X	for (t = strtok(p, seps); t; t = strtok(NULL, seps)) {
>X
>X	/* forbid starting with a slash */
>X		if (*t == '/')
>X			*t = '.';
>X
>X	/* change dots to slashes except in column 1 */
>X		while ( (q = strchr(t+1,'.')) != NULL)
>X			*q = '/';
>X
>X	/* for final dot or slash, remove it, and shut off recursion */
>X		recurseflag = 1;
>X		if (t[l = (strlen(t) - 1)] == '/') {
>X			recurseflag = 0;
>X			t[l] = '\0';
>X		}
>X
>X		if (*t == '!')
>X			exclude (t+1, recurseflag);
>X		else
>X			include (t, recurseflag);
>X	}
>X
>X	reap();
>X
>X	return (0);
>X
>} /* parse() */
>X
>X
>X
>/*
>X * newnode,exclude, include, preclude
>X *
>X *	functions to maintain global linked lists:
>X *		incl	include this path in list of dirs to reap
>X *		excl	leave out this directory
>X			(each entry can be recursive or not)
>X *
>X *	the functions are:
>X *		include (text,rflag)	add to incl list
>X *		exclude (text,rflag)	add to excl list
>X *		preclude()		clear both lists
>X */
>struct filspec *
>newnode(text)
>char	*text;
>{
>X	struct filspec	*f;
>X
>X	if ( (f = (struct filspec *) malloc (sizeof(struct filspec))) == NULL ||
>X	    (f->name = malloc (strlen(text)+1)) == NULL)
>X		ouch ("%s: out of memory\n");
>X
>X	strcpy (f->name, text);
>X	return (f);
>}
>Xexclude(text, rflag)
>char	*text;
>int	rflag;
>{
>X	struct filspec	*f;
>X
>X	f = newnode(text);
>X	f->recurse = rflag;
>X	f->next = excl;
>X	excl = f;
>}
>include(text, rflag)
>char	*text;
>int	rflag;
>{
>X	struct filspec	*f;
>X
>X	f = newnode(text);
>X	f->recurse = rflag;
>X	f->next = incl;
>X	incl = f;
>}
>preclude()
>{
>X	struct filspec	*p;
>X
>X	while (incl != NULL) {
>X		p = incl;
>X		incl = incl->next;
>X		free (p->name);
>X		free (p);
>X	}
>X	while (excl != NULL) {
>X		p = excl;
>X		excl = excl->next;
>X		free (p->name);
>X		free (p);
>X	}
>}
>X
>X
>/*
>X * ouch(fmt, arg)
>X *	outputs an error message and exits with error status
>X */
>X
>ouch (fmt,arg)
>char	*fmt, *arg;
>{
>X	fprintf (stderr, fmt, progname, arg);
>X	exit (-1);
>}
>SHAR_EOF
>chmod 0664 lib.c ||
>echo 'restore of lib.c failed'
>Wc_c="`wc -c < 'lib.c'`"
>test 3567 -eq "$Wc_c" ||
>	echo 'lib.c: original size 3567, current size' "$Wc_c"
>fi
># ============= list.sample ==============
>if test -f 'list.sample' -a X"$1" != X"-c"; then
>	echo 'x - skipping list.sample (File already exists)'
>else
>echo 'x - extracting list.sample (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'list.sample' &&
># yenta's /usr/lib/news/reaplist function script file.
>.1 junk,alt.flame,control,comp.binaries,alt.sex.pictures.,alt.fractals.pictures
>.1 alt.desert-storm,alt.desert-shield,soc.motss,soc.culture
>1 soc.singles,alt.activism,alt.romance.chat
>.5 talk
>1 sci,!sci.skeptic
>2 sci.skeptic
>.5 comp.text,comp.sys.atari,comp.windows,comp.os.vms
>.5 comp.unix.sysv386
>.5 misc.jobs,misc.handicap
>2 soc,!soc.singles,!soc.motss,!soc.culture
>.5 rec,!rec.arts.anime,!rec.arts.animation,!rec.arts.movies,!rec.games.hack,!rec.radio.amateur,!rec.autos,!rec.arts.tv,!rec.audio,!rec.video,!rec.music,!rec.skydiving
>1 rec.music,!rec.music.synth
>3 rec.music.synth,rec.skydiving
>2 alt.tv
>2 rec.arts.anime,rec.arts.animation,rec.arts.movies,rec.radio.amateur,rec.autos,rec.arts.tv,rec.audio,rec.video
>3 rec.games.hack
>1 misc.headlines,misc.consumers,misc.kids,misc.legal
>1 alt.sex,!alt.sex.bondage,!alt.sex.pictures.
>1.5 comp,!comp.dcom.telecom,!comp.binaries,!comp.text,!comp.sys.atari,!comp.windows,!comp.os.vms,!comp.sys.att,!comp.sys.3b1,!comp.sources.3b1
>3 comp.dcom.telecom
>1 comp.binaries
>1 alt.sources
>1 news,!news.announce.newusers
>3 biz
>3 misc
>3 alt,!alt.sources,!alt.tv,!alt.desert-shield,!alt.activism,!alt.fractals.pictures,!alt.desert-storm,!alt.romance.chat
>3 comp.sys.att
>5 comp.sys.3b1
>5 comp.sources.3b1
>7 general
># up to here is "normal expire"... now let's get desparate
>.1 alt.sex.pictures
>.1 rec.arts.anime,rec.arts.animation,rec.arts.movies,rec.ham-radio,rec.autos,rec.arts.tv,rec.audio,rec.video
>.1 biz
>.1 talk
>.1 news
>.1 soc
>.1 sci
>.1 comp
>.1 alt
>.1 misc
># if we got here, we're in big trouble -- wipe it all!
>0 .
>SHAR_EOF
>chmod 0664 list.sample ||
>echo 'restore of list.sample failed'
>Wc_c="`wc -c < 'list.sample'`"
>test 1611 -eq "$Wc_c" ||
>	echo 'list.sample: original size 1611, current size' "$Wc_c"
>fi
># ============= main.c ==============
>if test -f 'main.c' -a X"$1" != X"-c"; then
>	echo 'x - skipping main.c (File already exists)'
>else
>echo 'x - extracting main.c (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'main.c' &&
>X
>/*
>X * main.c
>X *	main routine and globals for reap utility, version 1.1
>X */
>X
>X
>#include "reap.h"
>X
>X
>/* linked lists for included and excluded file specs */
>struct filspec
>X		*incl = NULL,
>X		*excl = NULL
>;
>char
>X		*scriptfile = SCRIPT,	/* path of function script	*/
>X		*progname,		/* argv[0] for errors		*/
>X		*newsdir = NEWSDIR	/* news root directory		*/
>;
>int
>X		verbose = 0,		/* verbosity (-v) flag		*/
>X		dryrun = 0		/* do-not-unlink (-n) flag	*/
>;
>time_t
>X		age,			/* unlink files older than this	*/
>X		now			/* current time			*/
>;
>X
>X
>/************************
>X *	MAIN FUNCTION	*
>X ************************/
>X
>main(argc, argv)
>int	argc;
>char	*argv[];
>{
>X	int		device,		/* device # containing newsdir	*/
>X			c,
>X			lineno = 0,
>X			summary = 0,
>X			thenfree,
>X			errflag = 0;
>X	long		wantblox;	/* desired free space		*/
>X	char		*p;
>X	static char	line[MAXLINE];
>X	FILE		*script;	/* file ptr for function script	*/
>X	struct stat	st;
>X
>X
>/* save argv[0] for error spewings */
>X	progname = argv[0];
>X
>/* parse args */
>X	while ( (c=getopt(argc,argv,"vsnf:")) != EOF)
>X		switch(c) {
>X		case 'n':
>X			++dryrun;
>X			break;
>X		case 'v':
>X			++verbose;
>X			break;
>X		case 'f':
>X			scriptfile = optarg;
>X			break;
>X		case 's':
>X			++summary;
>X			break;
>X		default:
>X			++errflag;
>X			break;
>X		}
>X	
>X	if (argc - optind != 1 ||
>X	    sscanf(argv[optind], "%ld", &wantblox) != 1 )
>X		++errflag;
>X
>X	if (errflag) {
>X		fprintf (stderr,
>X		    "usage: %s [-v] [-s] [-n] [-f funclist] freeblocks\n",
>X		    progname);
>X		exit (-1);
>X	}
>X
>X
>/* stat newsdir to find the number of the device it's on */
>X	if (stat(newsdir, &st))
>X		ouch ("%s: can't stat %s\n", newsdir);
>X	device = st.st_dev;
>X
>/* open function script file for reading */
>X	if ( (script = fopen (scriptfile, "r")) == NULL)
>X		ouch ("%s: can't read %s\n", scriptfile);
>X
>/* Record the current time, for deciding what gets the axe. */
>X	time (&now);
>X
>/* chdir to newsdir */
>X	chdir (newsdir);
>X
>/* record initial freespace if summary is desired */
>X	if (summary)
>X		thenfree = freeblox(device);
>X
>/* main loop ... process until goal reached or end of script */
>X
>X	while (freeblox(device) < wantblox &&
>X	    (p = fgets (line, MAXLINE, script)) != NULL )
>X		parse (line), ++lineno;
>X
>X	fclose (script);
>X
>X	if (summary) {
>X		c = freeblox(device);
>X		printf ("Freespace was %d, now %d.  Cleared %d.\n",
>X		    thenfree, c, c - thenfree);
>X		printf ("Stopped after line %d in %s\n", lineno,
>X		    scriptfile);
>X	}
>X
>/* return 0 if freespace goal was met at exit, 1 if not */
>X	exit (freeblox(device) >= wantblox ? 0 : 1);
>X
>} /* main() */
>X
>SHAR_EOF
>chmod 0664 main.c ||
>echo 'restore of main.c failed'
>Wc_c="`wc -c < 'main.c'`"
>test 2497 -eq "$Wc_c" ||
>	echo 'main.c: original size 2497, current size' "$Wc_c"
>fi
># ============= reap.8 ==============
>if test -f 'reap.8' -a X"$1" != X"-c"; then
>	echo 'x - skipping reap.8 (File already exists)'
>else
>echo 'x - extracting reap.8 (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'reap.8' &&
>.TH REAP 8 LOCAL
>.SH NAME
>reap - remove news articles as space needed
>.SH SYNOPSIS
>.B reap
>[-v] [-s] [-n] [-f scriptfile] freeblocks
>.SH DESCRIPTION
>.I Reap
>checks disk freespace and deletes netnews articles
>according to a flexible user-specified
>scheme until
>.I freeblocks
>minimum freespace is available.
>It does this by sequentially reading commands (expire functions) from a
>text file (function script), and executing them one at a time.
>.PP
>Each expire function consists of an age limit and a list of newsgroups
>to expire to that limit.
>Before executing
>Xeach new expire function, reap checks the freespace on the news spool
>device.  Reap will exit when either the desired freespace is attained, or
>the end of the function script file is reached (the latter case is considered
>an unsuccessful exit).
>By carefully designing the function script, a news administrator can
>automate a wide variety of expiring policies.
>.PP
>The
>.B -n
>causes a dry run (as in make(1)).  When this
>option is specified, reap merely reports which articles it would delete if
>the option were left off.  This is useful for debugging function scripts.
>However, since no space is actually freed in this mode,
>it will either do nothing
>(there was enough space to begin with) or proceed through the entire function
>script file, indicating that reap would delete all applicable articles.
>.PP
>With the
>.B -v
>option, reap prints a verbose commentary on its progress to the standard
>output.
>.PP
>The
>.B -s
>option causes reap to print a brief summary of blocks freed to the standard
>output just before exiting.  It also indicates how much of the script file
>was processed.  This is independent of -v.
>.PP
>By default, reap looks in /usr/lib/news/reaplist for its list of functions.
>The
>.B -f
>option is used to specify an alternate function script file.
>.SH FUNCTION SCRIPT FILE FORMAT
>A function script file consists of a series of expire functions, one per line.
>Each expire function contains an age limit (in days, decimals okay), followed
>by a comma-separated list of newsgroup specifications, similar to the
>sys file format.
>.PP
>There are four types of valid newsgroup specifications that can go in the list:
>.PP
>(1) An ordinary newsgroup name (alt.sex) indicates that any articles in that
>newsgroup and any of its descendants (such as alt.sex.pictures), which are
>older than the age limit, should be expired.
>.PP
>(2) A newsgroup name with a trailing dot (alt.sex.) indicates that any articles
>in that newsgroup, which are older than the age limit, should be expired.
>However, recursion is not implied.  Articles in descendants of that group
>(such as alt.sex.pictures) are not affected.
>.PP
>(3) A newsgroup name with a preceding exclamation point (!alt.sex.pictures)
>indicates that that newsgroup and all of its descendants should be excluded
>from the list.
>For example, "1 alt.sex,!alt.sex.pictures" means to expire everything subsumed
>under alt.sex to 1 day, but do not touch anything subsumed under
>alt.sex.pictures.
>.PP
>(4) A newsgroup name with both a preceding exclamation point and a trailing
>dot (!alt.sex.pictures.) indicates that that newsgroup should be excluded
>from the list, but its descendants should remain in the list.  For example,
>"1 alt.sex,!alt.sex.pictures." means to expire everything subsumed under
>alt.sex to 1 day, including the contents of any subgroups of alt.sex.pictures
>(like alt.sex.pictures.d), but do not touch the articles in the newsgroup
>alt.sex.pictures.
>.PP
>Blank lines and lines beginning with a "#" are ignored.
>A sample function script file:
>.PP
>.nf
>X	# my first function script file
>X	0.5	talk,junk,alt.sex.pictures.
>X	2	rec,!rec.games,!rec.humor
>X	2	misc
>.fi
>.PP
>In the example,
>if space is needed, reap
>Xexpires to .5 days the entire talk and junk
>hierarchies, and alt.sex.pictures (not touching any subgroups of
>alt.sex.pictures, such as alt.sex.pictures.d).
>If still more space is needed, the
>rec hierarchy (excepting all of rec.games.* and rec.humor.*) is expired
>to 2 days.  If freespace is still short, reap then expires all of misc
>to 2 days.
>.PP
>Note that, while it would be possible to consolidate the 2-day
>lines, leaving them separate makes it possible for reap to stop
>in between them if sufficient space is cleared.
>.PP
>A practical function script file would directly or indirectly
>include every group carried on the system, with some age limit.  Otherwise,
>the spool directory will inevitably overflow without operator intervention.
>.PP
>It is valid to specify a dot all by itself (.) in the newsgroup list field.
>This is taken to mean the entire spool directory hierarchy, so "14 ." means
>to expire all news to 14 days.  Be warned that reap must search the entire
>news hierarchy when this feature is used.  Still, it is probably a good
>idea to end all function script files with a "0 .", so that, in absolute
>desperation, all news would be removed.
>.SH FILES
>.TP 25
>/usr/spool/news
>News spool directory
>.TP 25
>/usr/lib/news/reaplist
>Default function list file
>.SH DIAGNOSTICS
>Returns zero if sufficient space was free at exit, 1 if the entire function
>script was executed without freeing enough space, and -1 on an error
>condition.
>.SH CAVEATS
>What constitutes a disk "block" is implementation-dependent.
>.PP
>History files are not updated.  That must be performed as a separate operation.
>.PP
>For best performance, exclude recursively (without the dot) whenever
>possible.  Otherwise, reap will have to search the excluded directory
>for any subdirectories which might not have been excluded, which is slow.
>.SH BUGS
>The use of the ustat() system call is not very portable, and your humble
>author isn't aware of its non-system-V equivalents.
>.PP
>Lines in the function file are limited to 1024 characters.
>.PP
>The "cleared" figure in the summary merely indicates the difference in
>freespace before and after the run, not necessarily the actual number of
>blocks liberated by reap.
>.SH AUTHOR
>Reap is freeware by David B. Thomas (dt at yenta.alb.nm.us).  You are free
>to copy, distribute, staple, bend, fold or mutilate this package to your
>heart's content.  Please email any enhancements or ideas for enhancements.
>SHAR_EOF
>chmod 0644 reap.8 ||
>echo 'restore of reap.8 failed'
>Wc_c="`wc -c < 'reap.8'`"
>test 6129 -eq "$Wc_c" ||
>	echo 'reap.8: original size 6129, current size' "$Wc_c"
>fi
># ============= reap.c ==============
>if test -f 'reap.c' -a X"$1" != X"-c"; then
>	echo 'x - skipping reap.c (File already exists)'
>else
>echo 'x - extracting reap.c (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'reap.c' &&
>X
>/*
>X * reap.c
>X *	contains the reap() function.
>X *
>X * Once the global linked lists incl and excl have been stuffed,
>X * reap() actually scans the filesystem for files that meet the specs
>X * and unlinks them if they are older than the age global variable.
>X */
>X
>#include "reap.h"
>X
>X
>reap()
>{
>X	struct filspec	*f;
>X
>X	for (f = incl; f != NULL; f = f->next) {
>X		if (verbose)
>X			printf ("scanning %s ...\n", f->name);
>X		dodir (f->name, f->recurse);
>X	}
>X
>} /* reap() */
>X
>X
>X
>dodir(name, rflag)
>char	*name;
>int	rflag;
>{
>X	/* the following can be overwritten safely during recursion */
>X	static struct filspec	*e;
>X	static struct stat	st;
>X	static struct dirent	*dp;
>X	static char		thisname[MAXFILENAME+1];
>X	/* the following must be preserved through recursion */
>X	char			*fullpath;
>X	int			eflag = 0;
>X	DIR			*dirp;
>X
>X
>/* open directory for reading */
>X	if ( (dirp = opendir(name)) == NULL)
>X		ouch ("%s: can't read directory %s\n", name);
>X
>X
>/* see if this directory is excluded.
>X * If it's excluded recursively, quit here.
>X * If it's excluded non-recursively, set a flag, so we won't consider
>X *	deleting any files in it, but we'll still explore subdirectories.
>X */
>X	for (e = excl; e != NULL; e = e->next)
>X		if (!strcmp (name, e->name))
>X			break;
>X	if (e != NULL) {
>X		if (e->recurse)
>X			return;
>X		else
>X			++eflag;
>X	}
>X
>X
>/* loop for each directory entry */
>X	while ( (dp = readdir(dirp)) != NULL) {
>X
>X	/* name might be exactly MAXFILENAME characters long, and thus
>X	 * might not be null-terminated.  Some insurance:
>X	 */
>X		strncpy (thisname, dp->d_name, MAXFILENAME);
>X		thisname[MAXFILENAME] = '\0';
>X
>X	/* skip dot and dotdot */
>X		if (!strcmp(thisname, ".") || !strcmp(thisname, ".."))
>X			continue;
>X
>X	/* build the full pathname of current object */
>X		if ( (fullpath =
>X		    malloc(strlen(name)+strlen(thisname)+2)) == NULL)
>X			ouch ("%s: out of memory\n");
>X
>X		sprintf (fullpath, "%s/%s", name, thisname);
>X
>X	/* try to stat the object */
>X		if (stat(fullpath,&st)) {
>X			fprintf (stderr, "%s: can't stat %s\n",
>X			    progname, fullpath);
>X			free (fullpath);
>X			continue;
>X		}
>X
>X	/* maybe recurse if it's a directory */
>X		if ( st.st_mode & S_IFDIR ) {
>X			if (rflag)
>X				dodir (fullpath, 1);
>X			free (fullpath);
>X			continue;
>X		}
>X
>X	/* it's a file ...  is this a non-recursively excluded directory?
>X	 * if so, there's nothing to do to this file
>X	 */
>X		if (eflag)
>X			continue;
>X
>X	/* leave it alone if this directory is excluded, or
>X	 * if it's new enough.
>X	 */
>X		if (eflag || st.st_mtime > age) {
>X			free (fullpath);
>X			continue;
>X		}
>X
>X	/* reap this file! */
>X		if (dryrun) {
>X			printf ("Would unlink %s\n", fullpath);
>X			free (fullpath);
>X			continue;
>X		}
>X		if (verbose)
>X			printf ("Unlinking %s\n", fullpath);
>X
>X		if (unlink (fullpath) == -1)
>X			fprintf (stderr,
>X			    "%s: cannot unlink %s\n", progname, fullpath);
>X
>X		free (fullpath);
>X
>X	} /* while */
>X
>X	closedir (dirp);
>X
>} /* dodir() */
>SHAR_EOF
>chmod 0664 reap.c ||
>echo 'restore of reap.c failed'
>Wc_c="`wc -c < 'reap.c'`"
>test 2845 -eq "$Wc_c" ||
>	echo 'reap.c: original size 2845, current size' "$Wc_c"
>fi
># ============= reap.h ==============
>if test -f 'reap.h' -a X"$1" != X"-c"; then
>	echo 'x - skipping reap.h (File already exists)'
>else
>echo 'x - extracting reap.h (Text)'
>sed 's/^X//' << 'SHAR_EOF' > 'reap.h' &&
>X
>/*
>X * reap.h
>X *	header file for reap utility
>X */
>X
>/* set MAXFILENAME to the maximum number of characters in a filename for
>X * your system.  Typically 14 or infinity, where infinity equals
>X * 256 characters. :-)
>X */
>#define MAXFILENAME	14
>X
>/* set NEWSDIR to the directory containing news on your system.
>X * Very commonly /usr/spool/news
>X */
>#define NEWSDIR		"/usr/spool/news"
>X
>/* set SCRIPT to the path of the default function script file
>X * Usually this is /usr/lib/news/reaplist
>X */
>#define SCRIPT		"/usr/lib/news/reaplist"
>X
>X
>X
>X
>#include <stdio.h>
>#include <string.h>
>#include <ctype.h>
>#include <time.h>
>#include <malloc.h>
>#include <sys/types.h>
>#include <sys/stat.h>
>#include <ustat.h>
>#include <dirent.h>
>X
>#define MAXLINE		1024		/* max len of line in script	*/
>#define SECINDAY	(3600 * 24)	/* seconds in a day		*/
>X
>/* structure for linked lists of included and excluded file specs */
>struct filspec {
>X	char		*name;
>X	int		recurse;
>X	struct filspec	*next;
>};
>X
>Xextern struct filspec
>X		*incl,
>X		*excl
>;
>Xextern char
>X		*progname,
>X		*scriptfile,
>X		*newsdir
>;
>Xextern int
>X		verbose,
>X		dryrun,
>X		optind
>;
>Xextern char	*optarg;
>Xextern double	atof();
>Xextern long	freeblox();
>Xextern time_t
>X		age,
>X		now
>;
>SHAR_EOF
>chmod 0664 reap.h ||
>echo 'restore of reap.h failed'
>Wc_c="`wc -c < 'reap.h'`"
>test 1195 -eq "$Wc_c" ||
>	echo 'reap.h: original size 1195, current size' "$Wc_c"
>fi
>exit 0
>-- 
>Bottom of stack = 0x40000
>Stack pointer   = 0x3fffe
>Don't push it!
---
 reply to : admerlev at faui43.informatik.uni-erlangen.de