Saturday, September 24, 2011

Don't test for exact equality of floating point numbers


A discussion came up on Guido von Rossum's Google Plus post. It comes down to the fact that 2.1 is not exactly represented as a floating point number. Internally it's 2.0999999999999996, and this causes unexpected behavior.

These kinds of issues often come up. The confusion is caused by treating floating point numbers as exact numbers, and expecting calculations with them to produce results meaningful down to the last bit.

People at Wolfram Research has spent a lot of effort trying to get arithmetic right, and here's what a principle kernel developer, Daniel Lichtblau, has to say about the issue of making floating point calculations deterministic:

It is not in any way "solvable", at least not by means accessible to us (which in some sense defines it as "not a problem"). Depends too much on alignment-handling vagaries of MKL libraries, ordering of operations in BLAS, and usage, or not, of extended precision registers. See also IEEE 754: a careful reading may shed light on how different results for the same computation can arise in compliant hardware/software, even on the same machine

In addition to ambiguity in IEEE 754, which gives slight differences between different floating point libraries, there's an issue of the same library giving slightly different results on reruns with same inputs and and same machine, because of run-time optimization by the processor

The solution that Wolfram Inc came up with is to treat the last 7 bits of IEEE doubles as unknown. When testing for equality, those bits are ignored. When printing a number, it chooses representation that gives a nice printout. For instance, Print[2.0999999999999996] will display 2.1

So here's the rule of thumb for IEEE 754 floating point numbers:

When checking for equality of floating point doubles, ignore the last 7 bits of the mantissa.

14 comments:

Roman V. Shapovalov said...

Just in case you haven't seen it yet:
http://download.oracle.com/docs/cd/E19957-01/806-3568/ncg_goldberg.html

Ali Orangzeb said...

How to load notMNIST on keras library

Anonymous said...

got 94.5% accuracy with 3 layers (all with RELU): 1024, 256, 64, applied dropout with prob=0.5

Gokul Ravi said...

nice blog
android training in bangalore
ios training in bangalore
machine learning online training

Gokul Ravi said...

useful blog
python interview questions
cognos interview questions
perl interview questions
vlsi interview questions
web api interview questions
msbi interview questions

Gokul Ravi said...

laravel interview questions
aem interview questions
salesforce interview questions
oops abab interview questions
itil interview questions
informatica interview questions
extjs interview questions

Gokul Ravi said...

sap bi interview questions
hive interview questions
seo interview questions
as400 interview questions
wordpress interview questions
accounting interview questions
basic accounting and financial interview questions

Unknown said...

People are really wanted to go with the flows and let have a perfect life while they are floating in nowhere and then this could help them out. https://www.nursingcapstone.org/nursing-student-portfolio/ will give you some amazing idea and very effective for the papers writing.

Arthur Mendoza said...

While you wanted to go with the flow you need to know that not every time you should follow someon and go on. go to site that I love for the helpful information and its really amazing.

Arthur Mendoza said...

People are always waited for this day. Yeah, I am talking about the new year day when people love the day and love to do new thing with new promises. https://www.writemylitreview.com/our-services/ here abut the writing tips.

Anonymous said...

Word press is a best Open Source CMS which allows it to be used free of cost. You can use it on any kind of personal or commercial website without have to pay a single penny for it. It is built on PHP/MySQL (which is again Open Source) and licensed under GPL.
wordpress interview questions

poni said...

You cannot just measure with the right equality when the things are really floating. Click here http://www.paschoolrequirements.com/pa-programs-in-maine/ to know about the maine program.

linda said...

We may have keep the balance on where we are working on but for that every task should be take into account. http://www.personalstatementpsychology.com/counselor-personal-statement-that-will-succeed-8-tips-to-bare-in-mind/ will help you out with some tips.

kate technologies said...

Nice blog..! I really loved reading through this article. Thanks for sharing such a amazing post with us and keep blogging...
Internships for College Students in hyd
it internships in Hyderabad
Student Internships in Hyderabad