Summary: Integrating C, Make, SCCS, Lint, etc.

[3929] Brian Gilstrap]

Well, about two weeks ago, I asked for suggestions on using Make, SCCS, C,
and Lint together and was forced to give many kudos to those who noted that I
said something like "...don't suggest that we use RCS instead of make..."
instead of saying somthing like "...don't suggest we use RCS instead of
SCCS..."  Oops :-). 

Anyway, here is a summary of the information I got.  There were several notes
of the form "Make has inference rules that allow it to get the *.c from the
s.*.c".  I was aware of this but did not make my questions clear, so I don't
blame anyone who responded so.  This is a fairly long article as just a
summary, so the uninterested can junk it if they haven't already.

I do want to thank everyone who replied.  The suggestions have been quite
helpful.

In addition, I'd very much like to discuss my basic design in hopes of
discovering its (probably major) flaws before I fully implement it.  I guess
comp.misc is the appropriate place?  If I don't get any raging opposition,
I'll probably post a description of my design and let everyone pick it to
pieces.  So, anyone who doesn't normally read comp.misc and wants to join in,
look for me there in a couple of days.  (If I've let a more appropriate
newsgroup idea fall through my sieve of a brain, just let me know).

The comments:
-------------------------------------------------------------------------------
From: cirrusl!paul ( Paul E. Black at Cirrus Logic Inc. )

]We use a command, tdelta, to keep directories of "released" or
]"current" code up to date when a change is delta'd.  Thus the code is
]always available for lint, make, debugging (via sdb/dbx), viewing
](does this function really do what I think it does?), etc.  We never
]worry about getting make to check out code for compilation, where to
]put it, etc.  We also have a command, tget, which wraps up the
]check-out-and-lock get.
]
]We keep things divided into three kinds of directories: the SCCS
]directory (where the s. files are kept), the release directory (where
]a copy of the current code is always kept and where global
]<integration?> compiles are done), and testbed directories (where work
]is being done).  Makefiles have full pathnames to the release
]directory for all files.
]
]To work on a file, the programmer does the following
]	cd to a testbed directory (usually each programmer has their own)
]	tget the file which
]		- gets a local copy (supplying the appropriate options
]			and building the long and obscure pathname to
]			the SCCS directory), and
]		- edits the local makefile so the file will be taken
]			from the local dir instead of the release dir.
]	edit the file
]	make, which
]		- recompiles the local file, and
]		- links it with the other files in the release directory.
]	test
]	(repeat edit/compile/test if necessary, of course)
]	lint
]	tdelta the file which
]		- edits the local makefile so the file will again be
]			taken from the release directory,
]		- deltas the file (prompting for comments and wrapping
]			up the delta command for user convenience), and
]		- gets a new copy of the file to update the release
]			directory.
]If appropriate, the programmer also begins a recompile (make) of the
]release area.
]
]
]We have used this basic system for about 8 years, and switched to RCS
]with little trouble.  We actually have alpha-test directories (which
]are updated by tget and tdelta and are the main versions) and release
]directories (which are updated by rget and rdelta and are branch
]versions).  In addition we have several Sun 3 and Sun 4 file servers,
]so we have a distribution system which automatically sends out a new
]copy of files to the different machines when a tdelta occurs.
]
]If any of this seems useful or interesting to you, I would be happy to
]talk with on the phone, send a copy of the paper describing it, etc.
]
]Paul E. Black          | UUCP: ...{allegra,ames,sun}!oliveb!cirrusl!paul
]CIRRUS LOGIC, Inc.     | Internet: cirrusl!paul at olivetti.com
]1463 Centre Pointe Dr. | Voice: (408) 945-8305 x 210
]Milpitas, CA 95035 USA |

------------------------------------------------------------------------------

Tom Chmara provides several opinions on how to use SCCS for large projects.
I've deleted names because I'm not positive that people want to be quoted
(sometimes slightly out of context).  If anyone wants to claim individual
comments, or would like me to reveal that they are the author, I'd be happy
to do so.  Just let me know.

From: uunet!bnr-fos!tpc (Tom Chmara)
]
]This isn't directly applicable...but here are a number of
]opinions, etc. on how to use SCCS for large projects...
]   Hope this helps...
]	---tpc---
]
]==============================================================================
]
]I've been using SCCS successfully for about a year now, and I think I've
]come up with a scheme that handles at least most of what you're concerned
]about.
]
]The main thing missing in SCCS seemed to be the ability to group files
]together into what I call a "version"; I believe you call it a "load".
]I built a few little shell scripts to support this function:
]
]	Name		Function
]
]	ctl 		Place a file "under control"
]	control		Place a whole directory's files "under control"
]	upd			Update (do SCCS "delta") on a file
]	mkversion	Make a new version of all the latest deltas
]	getv		Grab all the files for a particular version
]
]My development directory just has two subdirs hanging off it: SCCS for
]the s-files themselves, and V for the "version files", which are just
]the list of version/filename pairs it takes to build a version.
]For maintenance of an existing version, all you have to do is grab that
]version (getv), edit the program file to your satisfaction, update the file, 
]and edit the version file to reflect the program file's new version.
]
]Right now I do all my development in the development directory, but there's
]nothing that requires this.  You could have multiple developers with
]multiple versions all over the place if you wanted.  
]
]I do have to "recompile the world" with this scheme when I want to grab
]an old version, but the systems I have are small enough that I'm not
]burdened by this.  If I were, the way to handle it would be to save
]compressed tar files of the binaries around in the V directory.  Multiple
]developers could share the .o's by creating links to them to start with;
]anyone wanting to make a change in a particular module, could just delete 
]the particular .o link in his directory.
]
]==============================================================================
]
]>I DO know that both SUN & HP use SCCS at some stage of their development
]>efforts, so presumably maintenance of old software streams is a subject
]>near (though maybe not dear) to their respective hearts.
]
]Yes, Sun uses SCCS to manage its source files; however, they have no
]magic answers to the questions you ask....
]
]>1.  Scenario:  Building a new load.  Problem:  devoted developer has a
]>    module checked out for editing.  I don't WANT the new version; I want the
]>    old one.
]
]Sun's answer here is to say that when rebuilding SunOS, if anybody has
]anything checked out for editing, they lose - it gets blown away. 
]Crude, but effective....
]
]>2.  Scenario:  Doing maintenance.  A bug has been reported in existing
]>    (read:  non-current) software.  Do I have to rebuild (recompile) the
]>    world to be able to restore that environment?  Do I have to maintain
]>    a copy of the directory hierarchy for every released version?
]
]Sun's answer here is to maintain a copy of the directory hierarchy for
]every released version, although they may not be maintained on line.
]
]>2.  Scenario:  Doing development.
]
]Basically, except when the source tree has been "checkpointed",
]developers run the risk of having an inconsistent source tree.
]
]Long-term, some of these problems may be addressed by the Sun NSE
]product, once SunOS is brought under NSE....
]
]==============================================================================
]
]>1.  Scenario:  Building a new load.  Problem:  devoted developer has a module
]>checked out for editing.  I don't WANT the new version; I want the old one.
]>"-c" gets the old version, "-G" specifies a new path into which I dump the
]>file.  Ergo, I need a new copy of my source directory structure?  Or does every
]>file need an "alias" for loadbuilding?
]
]We created a unique user who held all the sccs directories, only two people
]even knew the password and THEY never did development in the account.  This
]account held several makefiles driven by a number of scripts.  One was for an
]individual to do a make in his own directory somewhere, using .ol files from
]from the sccs user's directories, only unique modified files would need to
]have .ol files in the local directory and these would superceed the others.
]Your particular problem was dealt with by a user having several local
]directories with a make in each, as was sometimes done.
]
]>2.  Scenario:  Doing maintenance.  A bug has been reported in existing (read:
]>non-current) software.  Do I have to rebuild (recompile) the world to be able
]>to restore that environment?  Do I have to maintain a copy of the directory
]>hierarchy for every released version?
]
]We did two things, wrote a script which saved away the version numbers after
]each build, and saved a copy of the binary and the symbol maps for every
]released build.  Most bugs reported could then be duplicated by running the
]binary, narrowed down using the maps, and then the appropriate versions
]obtained from the version logs.  Our save directories were only about 1meg
]each and we did not release dozens of outside versions, so it worked.
]
]>3.  Scenario:  Doing development.  Do I need to do development in the current
]>directory (i.e. parent to SCCS directory) and aggravate the loadbuilders (see
]>#1)?  What happens if two of us are doing development in the same directory?
]>I'd like to use the old (read:  loadbuilt) objects to avoid having to recompile
]>the world; so would s/he.  We're constantly going to see each other's sources.
]>This looks like a problem for the Makefiles...  Unless I actively sccs info
]>each subdirectory, I don't know who else is doing development -- and that
]>information itself is volatile...
]
]As above, NOBODY did development in the SCCS system.  (Actually, one of the
]make file/script systems did a 'rebuild system and update all .ol files'.  That
]was done after group approval and all development came to a halt while it ran).
]Two people NEVER did builds in the same directory, individuals solved their own
]problems and then checked them back in to the SCCS account, (via suid and a
]validation checking script).
]
]All this what happened on one project to ensure security and safety.  More than
]a dozen people of varying degrees of skill managed to do hundreds and perhaps
]thousands of builds over a number of years to produce what was a very large
]piece of software (for us) without once corrupting the system.  We were VERY
]militant about not being able to have an individual corrupt the system, either
]by accident or intentionally, in any way other than checking in a file with a
]bug in it.  If they did check in such a file there was no way they could do that
]without it being recorded exactly who did it in an un-forgeale manner.  It can
]work, it just takes some creative effort to produce a system.
]
]------------------------------------------------------------------------------
Mike Matlack writes:

]Last year I worked on a software project with 4 other people and we could
]not have managed without SCCS and make.  We used these for both source code
]and documentation control/management.
] 
]A couple of lessons learned in their use follow...
] 
]- It was very helpful to have the makefile generated automatically.  If you
]are on a sun, the SunPro version of make does this nicely with the
].KEEP_STATE: option placed in the makefile.  If you are not on a Sun, I
]believe there is a public domain program called makedepend that looks at
]your source code, determines the file dependencies, and generates a
]makefile.  It is also useful to have some automatic way of notifying people
]that the makefile has been updated.  I don't recall if we ever devised a
]good way.  I imagine you could have some dependency on the makefile in the
]SCCS directory, such that if the makefile is out of date, it is "sccs
]get"'ed, make is exited, and a message "rerun make" is echo'ed.
] 
]- One problem with the sccs/make usage is that there are typically x copies
]of all the source, object, and executable files, where x is the number of
]people on the development team.  This can cause problem if you're short on
]disk space.  One solution (we never got around to implementing...remember
]this is lessons learned:-) was to create a single source directory (aside
]from the SCCS directory) containing the most recent copies of the source
]and object files.  Each person would then set up a development directory,
]where they did their work.  This development directory, instead of
]containing copies of the source code, would contain soft links to files in
]the source directory.  (good idea, Pete!)  When a user wanted to edit a
]file, they would do a form of an "SCCS edit" and replace the link with the
]editable file.  This should work fine for source code.  If you plan on
]implementing the object files like this too, it'll take a little more to
]get it to work.
] 
]- People typically use sccs/make for source code only, but it is also very
]beneficial to use them for documentation management/control too.  In the
]project I worked on, the 5 team members were responsible for documentation
](requirements specs, design documentation, user's manual, etc.) as well as
]source code development.  We used LaTeX, which allows you to break up your
]document into separate files (we typically assigned a section to a file).
]We then used sccs/make and set up dependencies such as the document (.dvi
]file) depends on the sources (.tex files) and figures.  We then logically
]broke up the document, assigning people to be in charge of certain
]sections.
] 
]- One problem that always seemed to come up was that someone would take a
]file out to edit, forget to put it back, and then go on vacation for a
]week.  It was often helpful to place a message in .login files which listed
]the files out for edit (sccs tell -u) and stated "do you really need these
]files out".
] 
]Hope this helps.  Please send me a summary of your replies.
] 
]Mike Matlack, General Electric, Advanced Technology Laboratories, 
]Systems Lab, 145-2, Moorestown Corporate Center, Moorestown, NJ  08057
](609/866-6612) Arpa: mmatlack at atl.ge.com, mmatlack%atl.decnet at ge-crd.arpa 
]uucp: ...steinmetz!atl.decnet!mmatlack  
]
]------------------------------------------------------------------------------
Erik Baalbergen

]From: Erik Baalbergen <uunet!mcvax!cs.vu.nl!erikb>

]First of all, RCS isn't a replacement for make; it might be a replacement for
]SCCS.

Right you are Erik. :-)

]Using SCCS in combination with older ``makes'' is really awkward, since
]SCCS uses a prefix (s.) and ''make'' is rather suffix oriented.  Most of
]today's ``makes'' (BSD, S5, etc) are able to deal with SCCS gracefully,
]using the suffix .X~ to denote SCCS repositories for files with suffix .X.
]To achieve full coexistence of make and SCCS, the built-in or predefined set
]of make rules has been extended to include rules of the form
]	.c~.o:
]		$(GET) [don't now the options] $<
]		$(CC) $(CFLAGS) -c $*.c
]Instead of using the conventional makes, you might think of using any of
]the improved ``makes'', e.g. the PD version ``cake'' [Somogyi], which was
]posted to the net a while ago.  Instead of transforming the prefix into a
]suffix, ``cake'' allows you to specify file-name-matching meta rules, like:
]	%.o: s.%.c
]		....
]Moreover, ``cake'' offers a lot of features over make, including makefile
]compilation, preprocessing of makefiles, conditional rules, and
]dynamc dependencies.  The latter feature might be exploited in using cake
]in combination with, e.g., lint and C, since dependencies of a C file
]can be determined at ``make'' time:
]	file.o:  [[ ccincl file.c ]]
]Using dynamic dependencies in any conventional ``make'' needs a lot
]of (dirty) hacking, or requires a separate
]''make'' run (``make depend'', which might overwrite the original Makefile).
]
]Other new makes you might be interested in, are:
]	Edition-8 Make, Enhanced Make [Hirgelt], GNU Make,
]	AT&T's Nmake [Fowler], Mk [Hume], Kmake [Thomas], and
]	various Make front-ends and makefile generators.
]If you're interested, I will send you a copy of an (informal and probably
]incomplete) survey on makes and make front-ends.
]-- 
]Erik H. Baalbergen				    <erikb at cs.vu.nl>
]Vrije Universiteit / Dept. of Maths. & Comp. Sc.
]De Boelelaan 1081
]1081 HV Amsterdam / The Netherlands		tel. +31 20 548 8080
]
]------------------------------------------------------------------------------

Chip Salzenberg writes:

]From: ateng.ateng.com!chip (Chip Salzenberg)
]
]We use these tools.  The missing link in your case is probably GNU Make.
]Now that I've used it, I'll never go back.  It's wonderful.  For example,
]GNU Make knows how to call "get" to check out a source file temporarily,
]if it doesn't exist.  Or if it does exist, and the SCCS file is newer,
]GNU make calls "get" to check it out.
]
]And that's just one nice feature.  Two other enhancements that are worth
]the price of admission: "include" and "ifdef".  Need I say more?
]
]You can get GNU make from prep.ai.mit.edu, or from uunet.uu.net.  If you
]can't find it there, try posting to comp.sources.wanted with a statewide
]distribution.
]-- 
]Chip Salzenberg                   <chip at ateng.com> or <uunet!ateng!chip>
]A T Engineering                     "Designed with your mind in mind."
]
]------------------------------------------------------------------------------

MH Cox writes:

]From: MH Cox <andante!cox at garage.att.com>
]
]Recommend reading _Topics in C Programming_ by Kochan.  Here are a few
]of rules I live by:
]
]All project makefiles should have the following targets in them:
]
]lint:		# performs lint on all the source files
]clean:		# removes all by-products of a make
]clobber:	# removes all products and by-products of a make
]install:	# installs the software in a the global executable
]		#  directory
]
]Some other useful ones might be:
]
]unit_test:	# perform whatever automated unit tests can be
]		#  performed for the code in this source directory
]
]lprof:		# execute lprof utility on source code (see 3.1 manuals
]		#  for explanation of lprof utilities)
]
]profile:	# as above for profile utility
]
]
]Always use make macro variables instead of hard-coding command names
]and flags.  For example, use $(CC) instead of cc; use $(CFLAGS)
]instead of explicitly using compiler flags.  This allows you to easily
]to make globally changes (a useful one is to set $(CC) to lint, thus
]linting all you files).

]Michael H. Cox			ARPA:  cox at garage.nj.att.com
]AT&T Bell Labs			UUCP:  ihnp4!bentley!cox
]184 Liberty Corner Road		COMPU$ERVE:  76525,3703
]Rm 3N-D12
]Warren, NJ  07060
](201) 580-4548

]--------------------------------------------------------------------------
Doug Howe writes:

]From: ole!exp at beaver.cs.washington.edu (Doug Howe)
]
]We long for the simplicity of SCCS here(:-), we're deeply entrenched in DSEE
]now (it's ok, but slow).  However, in the past we were into SCCS.  A few tip
]and hints that I remember off the top of my head:
]
]   Look seriously into lint libraries.  If you can, require user to write
]code which can pass lint.  This may involve a lot of (void)ing, etc., but
]it sure nips alot of bug in the bud.
]
]   Get the dependancy list for each library, especially the "tunneled header-
]files" (see attached csh script).  It produces the list of dependancies as
]output from the c-preprocessor, however you get to format.  If you get stuck
]I have a filter, but it's not that great.
]
]   If you will ever spread out onto foriegn machines, think about shying away
]from special chars in names(except '_' which is fairly widely excepted), no
]numbers as the leading character and no uppercase.
]
]   I also have a tool which I wrote to identify the files delta'd in since the
]last time the tool had been run.  Normally, you can put dependancies within
]the makefile which can do this fetch ( file.c : SCCS/s.file.c -> sccs get).
]However, we didn't want to make inside the database and this gave us the
]ability to also collect groups of files changed over periods of time so that
]they could be ported to other platforms.  It was not very efficient, however
]it's a workhorse.  It's also pretty simple since I learn C while I wrote it.
]
]   Also, work hard to maintain your SCCS headers.  Periodically, you will want
]to check out each file to see if someone checked in a file with expanded a
]header (they will!), unfortunately I don't remember the message you get.
]Especially, if you can require unique static headers in the header files you
]could potentially track all parts which went into an executable.  If you
]need something like this I probably can mail you a copy.
]
]   I'd be interested to know what platform(s) you're running on, and what other
]info you get.
]
]****** promised attatched csh script ******
]#! /bin/csh -f
]
]#  This script recursively lists all files #included
]#  by a C source code file.
]#
]#  To run type: depend *.c
]
]#  It works by running the C preprocessor on the file(s), then using awk
]#  to find all the lines that have a '# 1 ' and a '.h' somewhere in them.
]#  This works since #include files are represented in the preprocessor
]#  output as (for example):
]#		# 1 "/usr/exp/include/slic.h"
]#  The script prints out only what is between the quotes.
]
]foreach i ($*)
]   echo $i | sed -e 's/\.c/.o	:/'
]   cc -E $i | bm "# 1 " | awk '{ print $3}' | sed -e '/\/usr\/include/d' \
]   -e 's/\.\/\.\.\/\.\.\/\.\.\/include/\$\(INCLUDE\)/' \
]   -e 's/\.\/\.\.\/\.\.\/include/\$\(INCLUDE\)/' \
]   -e 's/\.\///' -e 's/\"//g' -e '/\.c/d' | sort
]end
]********* cut here **********
]
]Good luck, feel free to write ( or drop me a line ), but remember it's been
]3 year since I was actively SCCSing.
]
]-- 
]Doug Howe
]Seattle Silicon
]3075 112th Ave NE, Bellevue, WA  98004
](206)-828-4422			UUCP:  ...!uw-beaver!tikal!ole!concorde
]
]------------------------------------------------------------------------------

Chris Purdom writes:

]From: cs.utexas.edu!uunet.uu.net!snark!hutch!rabbit1!purdom
]
]1. Database for the source code with all relevant sccs information (and
]   anything else you'd like)
]2. User interface to database that can update individual entries and do
]   sccs manipulations, and can also recreate the whole source directory 
]   structure.
]3. Global make tool, ideally based on the information in the database,
]   (e.g. it calls all entries named "makefile")
]4. lint targets in all makefiles, which can be called by the global make tool
]   above.
]Fun, excitement, and lots of worthwhile work.
]---
]
]Chris Purdom                                    Frith-Inle
]Rabbit Software Corporation			(215) 647-0440
]Seven Great Valley Parkway East
]Malvern, Pennsylvania 19355
]
]

Well, that about covers it.  Thanks for the responses everyone.

Brian R. Gilstrap                          Southwestern Bell Telephone
One Bell Center Rm 17-G-4                  ...!ames!killer!texbell!sw1e!uucibg
St. Louis, MO 63101                        ...!bellcore!texbell!sw1e!uucibg
(314) 235-3929
#include <std_disclaimers.h>