Open Bug 1440184 Opened 7 years ago Updated 2 years ago

Does mfbt/tests/TestFloatingPoint.cpp (...and other files) need a special MSVC compile flag for correct double math?

Categories

(Core :: MFBT, enhancement)

58 Branch
enhancement

Tracking

()

People

(Reporter: Waldo, Unassigned)

References

Details

Bug 744965's first patch landing got backed out because MSVC in at least *some* configurations compiles

  pow(2.0, -1075.0)

(whose exact mathematical value is half of the smallest representable IEEE-754 double precision value, which is 2**-1074) to equal instead the mathematical value 2**-1074.  That is, with MSVC these expressions are both equal:

  pow(2.0, -1074.0)
  pow(2.0, -1075.0)

This doesn't appear to be ascribable to rounding mode, which per fegetround() is 0 which is FE_TONEAREST -- to the nearest value, and to even if there are two nearest values (and 2**-1074 has a low-order bit of 1 while +0.0 has a low-order bit of 0).

Best guess is maybe this file needs to compile with /fp:strict?  But if that's the case, why can't I find any mention of /fp:strict anywhere else in our tree?  We certainly do have other code that depends on accuracy of this stuff, and it's strange that this code would somehow be the first to stumble across this.

But in any case, the code inflicted with a possible compiler issue here is test-only, so it seemed best to land it with a hackaround -- then to fix it, whatever that entails, in a followup bug.  Wherefore, this bug.
This seems to be a bug in the pow.asm that ships in MSVC's libraries. I can reproduce this on 64-bit VS2015 and VS2017 (and also clang-cl when using it with VS2017 libs), but not 32-bit.

I stepped through and saw 2.0 and -1075.0 going in via xmm0 and xmm1, and 4.94066e-324 came back out in xmm0. But I don't speak IEEE-754 well enough to be able to tell you exactly what step went wrong.

I doubt any compiler flags will be able to help us here. Can you report this to the VS team?
Clarification... My test program was using pow.asm from MSVC libs because I forgot to add -MD for dynamic linking the CRT. I just tried again with -MD, and ucrtbase!pow has the same problem.
I filed https://developercommunity.visualstudio.com/content/problem/210143/pow20-10750-half-the-smallest-positive-nonzero-dou.html for this and suggested maybe they could add a special-case -- our exponent-is-int32_t special-case, maybe? -- to address it.  Not particularly optimistic on their fixing it, but of course from our point of view this isn't really the end of the world if they don't.
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.