Summary
Decimal.round/3 rounds twice for any value whose coefficient has more
significant digits than the context precision. The first rounding uses
Context.get().rounding; only the second uses the mode passed to round/3.
When the two disagree the result differs from a single correct rounding,
including a :down round returning a value larger than its input.
Reproduction
Under a reduced-precision context:
Decimal.Context.set(%Decimal.Context{precision: 3})
Decimal.round(Decimal.new("0.4995"), 1, :down)
#=> Decimal.new("0.5")
# expected Decimal.new("0.4") — :down truncates toward zero
The context's :half_up carries 0.4995 to 0.500, and :down then
truncates that.
The leak runs in both directions, truncating first under :floor discards
the nonzero digits :ceiling needs to see:
Decimal.Context.set(%Decimal.Context{precision: 3, rounding: :floor})
Decimal.round(Decimal.new("0.4001"), 1, :ceiling)
#=> Decimal.new("0.4")
# expected Decimal.new("0.5")
Reachability
At the default context the parse limit (max_digits: 34) and the precision
(34) coincide, so an over-precision literal never survives Decimal.new/1:
Decimal.new("0.49999999999999999999999999999999995")
** (Decimal.Error) : number parsing syntax: "0.49999999999999999999999999999999995"
That check only guards the parsing path. Decimal.new/3 performs no digit
count, and a driver decoding a numeric column builds the struct from the
binary wire format rather than parsing a string.
coef = String.to_integer("4" <> String.duplicate("9", 33) <> "5")
Decimal.round(Decimal.new(1, coef, -35), 1, :down)
#=> Decimal.new("0.5")
# expected Decimal.new("0.4")
The practical exposure is code that deliberately selects a non:half_up
mode :down for truncation, :half_even for banker's rounding,
:floor/:ceiling for directional rounding. Those are the modes that disagree
with the :half_up first pass, and the ones financial code chooses on purpose.
Cause
round/3 normalizes its input, and normalize/1 applies the context on both
of its non-zero branches. context/5 rounds the coefficient to
ctx.precision under ctx.rounding — so by the time do_round/5 sees the
value and applies mode, it has already been rounded once under a mode the
caller never asked for.
Not a regression
Long standing, not introduced by #240 or #241.
I have a fix, but would like guidance first
I have a working change locally, and am happy to send it as a PR,
but it runs into two questions that require a decision.
1. Is the current behavior intended? If round/3's contract is "apply the
context, then round to places", then this is documentation rather than a bug
and should be left alone. My reading is that it should be "round to places
under mode", with the context applying to the result like it does for every
other operation.
2. What should round/3 signal? The natural fix stops the input from going
through the precision pass, which also stops it setting :inexact/:rounded
for an over-precision input. Note round/3 doesn't currently signal for the
digits it discards Decimal.round(Decimal.new("1.234"), 1) sets no flags
so the flags fire today only as a side effect of input width. The options:
- signal for neither (what the minimal fix does)
- signal whenever
do_round/5 discards nonzero digits, which means threading
that back out of do_round/5, a wider change, but arguably the correct
one given round/3 is the operation closest to the spec's quantize
- leave the flags as they are, if the current behavior is depended on
Side Note:
The exponent limits have to stay applied to the input. do_round/5 scales the coefficient
by pow10(exp - target_exp) when exp > target_exp, which is unbounded for an
exponent built through new/3. Simply replacing normalize/1 with a
context free zero strip would let Decimal.round(Decimal.new(1, 1, 10_000_000), 0)
build a ten-million-digit coefficient before the context discarded it.
Let me know what you think.
Summary
Decimal.round/3rounds twice for any value whose coefficient has moresignificant digits than the context precision. The first rounding uses
Context.get().rounding; only the second uses themodepassed toround/3.When the two disagree the result differs from a single correct rounding,
including a
:downround returning a value larger than its input.Reproduction
Under a reduced-precision context:
The context's
:half_upcarries0.4995to0.500, and:downthentruncates that.
The leak runs in both directions, truncating first under
:floordiscardsthe nonzero digits
:ceilingneeds to see:Reachability
At the default context the parse limit (
max_digits: 34) and the precision(34) coincide, so an over-precision literal never survives
Decimal.new/1:That check only guards the parsing path.
Decimal.new/3performs no digitcount, and a driver decoding a
numericcolumn builds the struct from thebinary wire format rather than parsing a string.
The practical exposure is code that deliberately selects a non
:half_upmode
:downfor truncation,:half_evenfor banker's rounding,:floor/:ceilingfor directional rounding. Those are the modes that disagreewith the
:half_upfirst pass, and the ones financial code chooses on purpose.Cause
round/3normalizes its input, andnormalize/1applies the context on bothof its non-zero branches.
context/5rounds the coefficient toctx.precisionunderctx.rounding— so by the timedo_round/5sees thevalue and applies
mode, it has already been rounded once under a mode thecaller never asked for.
Not a regression
Long standing, not introduced by #240 or #241.
I have a fix, but would like guidance first
I have a working change locally, and am happy to send it as a PR,
but it runs into two questions that require a decision.
1. Is the current behavior intended? If
round/3's contract is "apply thecontext, then round to places", then this is documentation rather than a bug
and should be left alone. My reading is that it should be "round to
placesunder
mode", with the context applying to the result like it does for everyother operation.
2. What should
round/3signal? The natural fix stops the input from goingthrough the precision pass, which also stops it setting
:inexact/:roundedfor an over-precision input. Note
round/3doesn't currently signal for thedigits it discards
Decimal.round(Decimal.new("1.234"), 1)sets no flagsso the flags fire today only as a side effect of input width. The options:
do_round/5discards nonzero digits, which means threadingthat back out of
do_round/5, a wider change, but arguably the correctone given
round/3is the operation closest to the spec'squantizeSide Note:
The exponent limits have to stay applied to the input.
do_round/5scales the coefficientby
pow10(exp - target_exp)whenexp > target_exp, which is unbounded for anexponent built through
new/3. Simply replacingnormalize/1with acontext free zero strip would let
Decimal.round(Decimal.new(1, 1, 10_000_000), 0)build a ten-million-digit coefficient before the context discarded it.
Let me know what you think.