Skip to content

Commit

Permalink
Correctly round double precision sqrt (#256)
Browse files Browse the repository at this point in the history
As discussed in JuliaLang/julia#43786, openlibm's sqrt function is incorrectly rounded for i387. IEEE requires correct rounding for these functions and LLVM relies on it. Fix that by setting the precision in the FPU control word (see e.g. e_ceil.S for similar FPU modifications).
  • Loading branch information
Keno authored Jan 19, 2022
1 parent 81d5e16 commit ae2d916
Showing 1 changed file with 19 additions and 2 deletions.
21 changes: 19 additions & 2 deletions i387/e_sqrt.S
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,29 @@
//__FBSDID("$FreeBSD: src/lib/msun/i387/e_sqrt.S,v 1.10 2011/01/07 16:13:12 kib Exp $")

ENTRY(sqrt)
fldl 4(%esp)
pushl %ebp
movl %esp,%ebp
subl $8,%esp

fstcw -4(%ebp) /* store fpu control word */
movw -4(%ebp),%dx
andw $0xfeff,%dx /* Set precision field to 64 bits (53 bit mantissa).
We assume it's set to 0b11 (extended precision),
so zeroing out the low bit of the precision field,
will correctly set the precision */
movw %dx,-8(%ebp)
fldcw -8(%ebp) /* load modfied control word */

fldl 8(%ebp)
fsqrt

fldcw -4(%ebp) /* restore original control word */

leave
ret
END(sqrt)


/* Enable stack protection */
#if defined(__ELF__)
.section .note.GNU-stack,"",%progbits
Expand Down

0 comments on commit ae2d916

Please sign in to comment.