SPARC calling conventions & coroutines

[boulder!foobar!grunwald at ncar.ucar.edu]

The following is a question concerning the SPARC architecture & the SunOS
support for the architecture.

I'm trying to port my C++ based threads package to a SPARC (Solbourne).
This is part of a multiprocessor simulation package for process oriented
discrete event simulation, written in G++. This code works on the
DECstation-3100, 68K based units, and in parallel on the Encore Multimax
(NS32K). So, I'm pretty certain the machine independent part works.

Threads are implemented via co-routines & private stacks. No stack copy is
done (too slow). The heart of the context switch on a SPARC looks like:

void
HardwareContext::magicSwitchTo(HardwareContext *to)
{
    asm("ta	%0" : : "I" (ST_FLUSH_WINDOWS));
    asm("st	%%sp,%0" : : "m" (sp));
    asm("st	%%fp,%0" : : "m" (fp));
    asm("ld	%0,%%sp" : : "m"(to -> sp));
    asm("ld	%0,%%fp" : : "m"(to -> fp));
    asm("ld	[%sp+60],%i7");	// prep return address
}

a ``HardwareContext'' represents a coroutine. ``to'' is the coroutine
to switch to. In assembly, this becomes:

.global _magicSwitchTo__15HardwareContextPT0
	.proc 1
_magicSwitchTo__15HardwareContextPT0:
	save %sp,-112,%sp
	ta	3
	st	%sp,[%i0]
	st	%fp,[%i0+4]
	ld	[%i1],%sp
	ld	[%i1+4],%fp
	ld	[%sp+60],%i7
	ret
	restore

An initial 3 stack contexts is built (by me), causing the initial ret to
``return'' to a thread entry routine; subsequent returns return to the
calling point.

I've read the SPARC architecture book that comes with the Sun doc's, but
have been having problems. I'm not entirely certain it's not related to
the g++ compiler I'm using, but...

The questions I'm after deal mainly with the way the FLUSH_WINDOWS trap
works, and when it's invoked, as well as the window restore/save code.

>From the SPARC calling convention, I gather that on a save trap, the IN
and LOCAL registers are saved relative to the current SP, and that
preceding windows are saved by following the call chain and saving
relative to the SP.

QUESTION 1: when does this stop? i.e. if I ``fake'' call chains, how do I
   tell the save/restore code that this is ``the top of the stack'' and that
   no more window saves/restores should be done?  This isn't defined in the
   arch manual.

QUESTION 2: Does this imply that the current IN registers are also written
   to memory, or just previous IN registers? I'm assuming they are, which is
   why the ``prepping the return address' comment is there -- I need to pull
   the return address from the saved IN to return to the other thread. It
   doesn't work any differently if i put a  ``st %i7,[%sp+60]'' right after
   the ``ta 3'', so I'm assuming that this is already done.

QUESTION 3: I notice that setting breakpoints appears to flush the current
   IN and LOCAL's to memory, relative to %sp. Is this correct?  (I noticed
   this when single-stepping through the coroutine switch)

QUESTION 4: If the current IN window is flushed at breakpoints, when else
   is it flushed? e.g., at UNIX context switch time? If so,  than the
   previous code is bogus, because once I set the %sp, I might over-write the
   saved state of the ``to'' context. Not that the ``to'' context would use
   the in's or locals (other than %i7, the return address).

If anyone knows the answers of 1..4, please mail me. In particular, if
you've done something similar, please let me know -- my code seems to
work, but after >60K context switches, something goes bonkers. I'm curious
about question 4, because it might be that the ``to'' return address is
being overwritten on a UNIX process switch. That would be very annoying,
because it would require a more elaborate context switch mechanism of
pointing %sp to a ``dummy'' stack area, explicitly loading %i0..%i7 and
then setting %sp to the ``proper'' stack area.

Dirk Grunwald -- Univ. of Colorado at Boulder	(grunwald at foobar.colorado.edu)
					(grunwald at boulder.colorado.edu)