RMS deviation between two sets of cartesian coords

[James F. Blake]

>From article <14660 at smoke.brl.mil>, by gwyn at smoke.brl.mil (Doug Gwyn):
> In article <27599 at cs.yale.edu> jim at doctor.chem.yale.edu (James F. Blake) writes:
>>     I am looking for code to compute the RMS deviation between two
>>sets of cartesian coordinates.  The coordinates would be arbitrarily
>>oriented with respect to each other, so some sort of fitting would
>>be required (LLSQ, NLLSQ, or simplex).
> 
> Sounds to me like the sums of the squares of the (vector) differences
> between corresponding coordinates would suffice -- or is it the
> situation that you can't establish a correspondence between points
> in the two sets (i.e. you want to compare clouds as such).

     That was the approach I took first.  It turns out that their is a
significant amount of "round off" error inherent in the least-squares
equations when solved in this manner.  If I computed the RMS deviations
between structures A -> B and B -> A, I would see as much as a 5%
difference in answers.  I was hoping that someone else had seen this
problem and decided to solve for the rotation/translation matrix
of [X][CTM] = [X'] directly.

  Jim