sun-3 dbx, arguments, hacking, help... (LONG)

[R. Nigel Horspool]

Here is an answer to the question about determining how
many bytes of parameters have been passed to a C function
executed on a SUN-3 (i.e. a Motorola 68000).
I don't know if this is close to how SunOS 4.0 dbx does it, but
in the absence of symbol table information, I can't see any other
way it could be done.

I have presented my answer in tutorial form so it's rather long.

First, let's modify the example code we were given so that function
`go' has some parameters and local variables.

>>	go( a, b, c )
>>	int a, b, c;
>>	{
>>	    int a[10];
>>
>>	    /*  most of the body of `go' is omitted  */
>>
>>	    do_anything(1, 'a', 2, "help");
>>	}

The prologue code at the entry to the `go' function has to create a new
stack frame to hold the local variables and link this stack frame to
the caller's stack frame.  Both actions are accomplished by the single
instruction called `link' on the 68000.

So, at the beginning of the code for `go', there will be an instruction:
		link  a6,#-40
where 40 is the number of bytes of local storage for `go'.  The minus
sign appears because the stack on which the memory is allocated grows
downward in memory.  (If the -O option hasn't been used, an equivalent
pair of instructions:  link a6,#0  addw #-40,sp  may appear instead.)

Immediately before calling `do_anything' (i.e. after pushing the 4
parameters in reverse order) and again immediately after returning from
that function, the stack will look like

	a6  ------>	|          |
			|----------|
			| 40 bytes |
			| for the  |
			| locals   |
			|----------|
			| arg  #4  |
			|----------|
			| arg  #3  |
			|----------|
			| arg  #2  |
			|----------|
	sp  ------->    | arg  #1  |
			|----------|

Therefore a computation to determine how many bytes of parameters are
on the stack at either of these points in the execution is:
	(a6) - 40 - (sp)
But this code would need to be executed by the caller of `do_anything'
and that isn't very useful.

If you want to execute code inside `do_anything' that determines how
many bytes of parameters were passed to it, the diagram is more
complicated (like a double version of the diagram above).  Let's assume
that `do_anything' has 20 bytes of local variables.  Then the picture
after `do_anything' has allocated its frame looks like:

			|          |  <----
			|----------|       |
			| 40 bytes |       |	(stack frame
			| for the  |       |	 for `go')
			| locals   |       |
			|----------|       |
			| arg  #4  |       |
			|----------|       |
			| arg  #3  |       |
			|----------|       |
			| arg  #2  |       |
			|----------|       |
			| arg  #1  |       |
			|----------|       |
			| ret addr |       |
			|----------|       |
	a6  ------>	| dyn link +------>
			|----------|
			| 20 bytes |		(stack frame
			| for the  |		 for `do_anything')
	sp  ------>	| locals   |
			|----------|

A calculation for the number of bytes of parameters from inside
`do_anything' is:
	((a6)) - (a6) - 48
where parentheses indicate dereferencing (i.e. following a pointer),
and the 48 is composed from 40 bytes for the caller's local variables
plus 4 bytes for the return address pushed on the stack by the function
call instruction (jsr) plus 4 bytes for the dynamic link pointer pushed
on the stack by the link instruction.


If we knew the number of bytes of local variables in the caller, it
would be easy to solve the problem in a couple of lines of C.
But finding the number of bytes of local variables in the caller ???
That's a bit harder.  The only technique I can think of is...

(1) Get the address of the caller's stack frame.
(2) Use the return address value in the caller's frame to find the
    `jsr' instruction that called the caller.
(3) By inspecting the operand of the `jsr' instruction, find the
    entry address of the caller.
(4) Finally, by looking at the immediate operand of the `link'
    instruction at the caller's entry point, determine the number
    of bytes of locals in the caller.

It's all highly dependent on the style of code emitted by the SUN C
compiler, but below is an example C program which performs all the
necessary calculations.  Remember, the program will produce the right
outputs only on a SUN-3 and only when it is compiled with the -O flag.


		R. Nigel Horspool
		University of Victoria
		(that's in Canada)


----------------- sample program follows -------------------

    /* foo determines how many bytes of parameters it has
       been passed on a SUN-3 (a 68000-based machine) */
    void foo( a )
    int a;
    {
        int *cheat[1];	/* `cheat' must be the FIRST local variable */
        int  *p, nbytes;
        short *q;
        /* declarations for other local variables appear here */
        
        p = cheat[1];		/* address of caller's stack frame */
        nbytes = ((int)p - (int)(cheat) - 12);
        p = (int *)(p[1]);	/* load return address */
        q = (short *)(p[-1]);	/* load entry address from jsr */
        nbytes += q[1];
        
        printf( "Number of bytes of parameters == %d\n", nbytes );
    
        /* code for foo can continue here */
    }
    
    main() {
       int a[10];
       foo();			/* pass 0 bytes */
       foo( 1 );		/* pass 4 bytes */
       foo( 2, 4 );		/* pass 8 bytes */
       foo( 3, "abc", 4, 5 );	/* pass 16 bytes */
    }

------------------------ end of sample program --------------------