Floating Point Expectations

Eliot Moss moss at cs.umass.edu
Sat Jun 2 04:40:00 AEST 1990


Just a minor followup to the (45.0 / 100.0) == .45 question:

While the integers 45.0 and 100.0 are undoubtedly represented exactly, .45 is
a repeating form in binary (just as, say, 1/7 is in decimal notation). One
*would* expect (50.0 / 100.0) == .50 since .50 *does* have an exact
representation in binary. In general, when writing floating point algorithms
one must demand equality only within a tolerance; choosing that tolerance is a
complex issue, and one can have relative and absolute tolerances. Hope this
adds a little to the original posters understanding ....		Eliot
--

		J. Eliot B. Moss, Assistant Professor
		Department of Computer and Information Science
		Lederle Graduate Research Center
		University of Massachusetts
		Amherst, MA  01003
		(413) 545-4206; Moss at cs.umass.edu



More information about the Comp.unix.wizards mailing list