Interprocess communication using pipes; too much buffering

[Jari Junikka]

Machine and Unix: AT&T 3B15 and 3B2/400 Unix SYS V rel 3.1

Goal: Cayley is a symbolic algebra system for group theory.
      I want to extend Cayley by writing a C-program, which
      1. Reads a line from the user
      2. Checks whether the line contains any non-Cayley commands.
      3. If not, passes the line to Cayley, waits for Cayley to
	 complete the command, and reads the output from Cayley
	 and prints the output.
      4. If the line contains non-Cayley commands,
	 a C-routine implementing the non-Cayley command
	 is called. This routine gives commands to Cayley,
	 reads the output, processes it, and so on,
	 until the desired task is completed.

Problem: I tried to use Cayley as a coroutine by redirecting
it's input and output to pipes. First attempt looked like:
(this is to give you the idea how I tried to do it,
it's obviously not the whole code... I do check error returns,
close unnecessary ends of the pipes etc.)

pipe(to_cayley);
pipe(from_cayley);
if ((child=fork())) then /* parent */
   /* loop ask for line, write to pipe, read from pipe, write to stdout */
else /* child */ {
   close(0); dup(to_cayley[0]); close(1); dup(from_cayley[1]); 
   execlp("cayley","cayley",NULL); 
}

Bug: read will not return anything before some system buffer becomes full!
Question: Is there anyway to get the output from Cayley immediately available?

I spent a night reading man pages, programmers reference manuals etc.
(Among other things I tried setting O_NDELAY by fcntl in various combinations
 of the file descriptors. Best result: I got the Cayley output immediately,
 but after reading all the available input, the process died: nothing in
 pipe--> Cayley thinks EOF --> exit.) 

The next day I looked for reference in desparation:
I brought Advanced Unix Programming by Marc J. Rochkind.
Second attempt according to example about interactive search command
using ed, (pages 151-154). The ed thing works, after I modified it
for cayley, the output from coroutine (Cayley) is not to be seen. 
Cayley output becomes available 1) if some buffer gets full 
2) the process running Cayley terminates. 

This version looks like:

static FILE *sndfp, *rcvfp; /* streams to read/write to Cayley */

static invoke_coroutine()
{
  pipe(pfdin); pipe(pfdout);
  switch (fork()) {
  case -1: /* failed */
  case 0: /* the child */
    /* do the usual play to redirect the stdin and stdout to pipes */
    execlp("cayley","cayley",NULL);
  }
  /* the parent */
  sndf=fdopen(pfdout[1]);
  rcvfp=fdopen(pfdin[0]);
}

main() { 
  invoke_coroutine();
  /* read line from user, write it to sndf using fputs, */
  /* write a command giving recognizable, unique output (fputs) */
  /* read (using fgets) rcvfp until the the unique output is met, loop */
}

This has the same problem: I get to see output from Cayley
after 1) giving commands which generate a lot of output, or
2) it terminates. 

I tried to prevent the output buffering  by adding a line
  setbuf(stdout,NULL);
just before execlp in invoke_coroutine, BUT this leads to the same
problem I had with read/write and fcntl. I get the full output
for all the commands I give before entering the interactive loop,
but the child process terminates after it has performed all the
commands given before entering this loop! Why does setting the stdout
buffer to zero affect the stdin Cayley is reading? Does it?
My reasoning is, that Cayley tries to read from empty pipe
and instead of blocking decides stdin=EOF and terminates.

The difference in the behaviour of ed and cayley in the last
attempt leads me to believe the way they write to stdout is
fundamentally different (simple guess: ed flushes it's output).

I'm lost, any ideas would be appreciated.

junikka at math.wisc.edu