Sun serial port communication problem

[dan at watson.bbn.com]

I've never tried to connect two serial ports together on Suns, but I can
give some general UNIX advice about connecting two tty ports together:

First, make sure those getty processes really went away when you wanted
them to.  Do "ps ax" and make sure there are no processes living on ttya
or ttyb, particularly processes named '-'.  They can cause intermittent
failures if they are still hanging around.

But your real problem is probably the tty modes.  The tty drivers are
capable of doing quite complicated things to the characters they send and
receive, and you don't want them to do any of that.  I did

	stty everything >/dev/ttya

on my Sun-3/50 running 3.5, to see what the default modes were, and the
result would not be good for your application: both echo and -nl appeared.
This sounds like it's the real problem.  The doubling of your newline
characters implies that -nl, called CRMOD at the programming level, is set
on both ttys.  So when you write an LF to one tty, CR-LF goes out on the
wire.  The reader sees CR-LF and turns it into LF-LF.  The repeating
garbage sounds like echo is turned on.  Actually, if you've got echo
turned on at both ttys, I'm not sure why your reader would ever stop
reading :-).  But you want to turn it off.

To do all this, use ioctl(2) to do a TIOCSETP as described in tty(4).  To
play it safe, do a TIOCGETP first to fill in the structure, then change
the bits you need to change and do TIOCSETP of the result.  Here is an
*untested* example:

	#include <sgtty.h>
	...
	struct sgttyb ttymodes;
	...
	/* Assume infd is the file descriptor being read */
	if (ioctl(infd, TIOCGETP, &ttymodes) == -1)
		perror("Error getting modes");
	ttymodes.sg_flags = RAW|TANDEM;
	if (ioctl(infd, TIOCSETP, &ttymodes) == -1)
		perror("Error setting modes");
	...	

In this example I've turned on raw mode, which is about as close to zero
processing by the driver as you can get.  All 8 bits will come through
uninterpreted.  And I turned on TANDEM, on the assumption that you want
the Sun to send ctrl-S/ctrl-Q characters back to the OGM if there's a
problem keeping up.  (Caveat: I don't know for sure that TANDEM works when
RAW is set...)  I also implicitly turned off things like CRMOD, ECHO, line
editing characters, etc. none of which you want.  If you want parity
checked/produced, turn on CBREAK instead of RAW.

You can also do this kind of thing at the shell level by using the stty
command; this is useful for your fake writer.  Example:
	(stty 9600 raw; cat a-file) >/dev/ttya
That sets the speed to 9600 baud and turns on raw mode.

Disclaimer: The details may not be quite right.  But it should get you
pointed in the right direction.

	Dan Franklin