diff --git a/README.md b/README.md index 878aad3..5995234 100644 --- a/README.md +++ b/README.md @@ -36,8 +36,9 @@ that is exact barring overflow and underflow. Two operations trade accuracy for divisions instead of three, within a proven 15 u² + 56 u³ barring underflow and overflow, and `OD::sloppy_mul` takes about 1/1.35 of the time of `*` and gives up 7 to 9 of the 424 bits. An octo-double meets a double-double through `OD::mul_dd`, `OD::add_dd` and `OdAcc::add_prod_dd`, which skip the zero parts' products and give the promoted -operation's bits, a NaN's sign and payload aside. The API documentation lists the rest, such as division by a shared -divisor and conversion to arrays of parts. +operation's bits, a NaN's sign and payload aside. `DD::new`, `QD::new` and `OD::new` (and `OV::new` on each lane) take +parts in any order and return their exact sum, largest part first, each part at most the ulp of the one before. The API +documentation lists the rest, such as division by a shared divisor and conversion to arrays of parts. ### Packed storage @@ -279,10 +280,10 @@ so octo-double values above about 2⁻⁶⁰⁰; near underflow the tests allow | `Ou` | derivatives within 10⁻¹²³, and its value is `OV`'s bit for bit | Every check can fail. The feature `mutate` breaks steps of the quad-double and octo-double algorithms, of exp, ln, sin, -cos and tan and their lanes, of the jets, dual numbers and field sums, and of the packed storage's decoders, one at a -time, and the tests run with it check that each breakage fails a test. The integration tests also feed deliberately -wrong results to the same checks: a double-double add without its low parts, a wrong derivative, a perturbed LU factor, -and f128 conversions that round twice. +cos and tan and their lanes, of the jets, dual numbers and field sums, of the packed storage's decoders, and of the +constructors, one at a time, and the tests run with it check that each breakage fails a test. The integration tests +also feed deliberately wrong results to the same checks: a double-double add without its low parts, a wrong derivative, +a perturbed LU factor, and f128 conversions that round twice. ## Testing, coverage and benchmarks diff --git a/python/README.md b/python/README.md index 69ec33e..22ccbf4 100644 --- a/python/README.md +++ b/python/README.md @@ -37,11 +37,11 @@ print(md.isa()) # the SIMD backend in use, detected at - `qd_axpy(a, x0, x1, x2, x3, y0, y1, y2, y3)`: y + a x for quad-double arrays x and y (normalized parts) and a float a, elementwise in quad-double; returns the four parts. - `od_solve(a0, ..., a7, b0, ..., b7)`: x with A x = b by LU with partial pivoting in octo-double (eight parts, - about 126 digits), A as eight (n, n) part arrays and b as eight of shape (n,) or (n, k); returns the eight parts of - x, to about cond(A) 2^-418. `ODLu(a0, ..., a7)` keeps the factors for several solves, `solve(b0, ..., b7, + about 126 digits), A as eight (n, n) part arrays and b as eight of shape (n,) or (n, k), each entry the exact sum of + its parts in any order; returns the eight parts of x, to about cond(A) 2^-418. `ODLu(a0, ..., a7)` keeps the factors for several solves, `solve(b0, ..., b7, transpose=False)`. -- `od_add_dd(z0, ..., z7, dh, dl)`: an octo-double array (eight float64 arrays, each element renormalized first) plus - a double-double one, elementwise in octo-double; returns the eight parts of the sum. +- `od_add_dd(z0, ..., z7, dh, dl)`: an octo-double array (eight float64 arrays, each element the exact sum of its + parts in any order) plus a double-double one, elementwise in octo-double; returns the eight parts of the sum. - `od_axpy(a, x0, ..., x7, y0, ..., y7)`: y + a x for octo-double arrays x and y (normalized parts) and a float a, elementwise in octo-double; returns the eight parts. - The array arguments of the functions above may be any arrays or sequences of real numbers, strided, in Fortran order diff --git a/src/any_order.rs b/src/any_order.rs new file mode 100644 index 0000000..70e7b62 --- /dev/null +++ b/src/any_order.rs @@ -0,0 +1,331 @@ +//! Parts in any order to their exact sum with the parts largest first, for the constructors `QD::new`, `OD::new` and +//! `OV::new`. The result is on the ladder (`on_ladder`): zero parts only after the nonzero ones, and each part at most +//! the ulp of the one before. The constructors take one of three paths (lsb and msb are the exponents of a number's +//! lowest and highest set bits): +//! +//! * Parts already on the ladder, as the arithmetic returns them, are renormalized as given, with the bits the +//! constructors always gave. Each part is at most 2^-52 of the one before, so each partial sum of VecSum, the first +//! pass, stays below the next larger part and its quick two-sums are exact. Each error of VecSum is a multiple of +//! the ulp of the partial sum it rounds and smaller than half the ulp of the partial sum above, so its nonzero +//! outputs do not overlap, and the second pass below returns them on the ladder. +//! * Other parts are sorted by decreasing magnitude (`by_magnitude`) and renormalized with error-free steps, which is +//! exact for N parts to N (paper/expansion-error-bounds.pdf, renormalization in any order). The result is kept when +//! it is on the ladder. Sorting alone does not guarantee that: parts that overlap in one binade can leave the second +//! part past the ulp of the first (`tests/any_order.rs`, the control without the fallback). +//! * Such a result, or one that overflowed although the parts are finite, is replaced by `expansion_sum`, on the ladder +//! for any finite parts. Grow-Expansion (J. R. Shewchuk, "Adaptive precision floating-point arithmetic and fast +//! robust geometric predicates", Discrete Comput. Geom. 18 (1997) 305-363, doi:10.1007/PL00009321) adds the parts +//! one by one into an expansion whose nonzero components do not overlap: the msb of each is below the lsb of the next +//! larger one. Every step is a two-sum, so the sum is exact in any order. Then the renormalization's second pass (the +//! first half of Shewchuk's Compress) runs from the largest component down. If that overflows, the parts are added +//! again, each next one the largest left of the sign opposite to the running sum, so that the running sum stays +//! within the largest part or the total and overflows only if the total does. +//! +//! The second pass adds each component w_j into an accumulator A by two-sum, emits the sum t as a part when the error +//! is nonzero, and carries the error on. On components that do not overlap it is exact and on the ladder. A is built +//! from larger components, so it is a multiple of 2^(lsb(w_j) + 1), and lsb(w_j) is the lsb of A + w_j. A step that +//! emits has an inexact A + w_j, whose msb is at least lsb(w_j) + 53, so 2^lsb(w_j) <= ulp(t)/2. The next part rounds +//! the carried error (at most ulp(t)/2) plus components below w_j (together below 2^lsb(w_j)), so it is at most ulp(t). +//! +//! With the cargo feature `mutate`, `mutate::set(id)` breaks the constructors on purpose in the current thread, for +//! the controls of `tests/any_order.rs`; without it the hooks are compile-time `false`. +use crate::dd::two_sum; +use std::cmp::Ordering; + +/// Deliberate breakage of the constructors `DD::new`, `QD::new`, `OD::new` and `OV::new` for the controls (feature +/// `mutate`; ids below, 0 = none). +pub mod mutate { + /// the parts renormalized as given, without the sort and the check: the constructors before they took parts in any + /// order (`DD::new` by a quick two-sum, `QD::new` by `renorm4`) + pub const AS_GIVEN: u32 = 1; + /// the sorted parts' renormalization kept when it is off the ladder or overflowed (no `expansion_sum`) + pub const NO_FALLBACK: u32 = 2; + /// `expansion_sum`'s result kept when it overflowed (no second Grow-Expansion with the signs alternating) + pub const NO_BALANCE: u32 = 3; + /// `OV::new` keeps the lanes' renormalization when its first part is not finite, also when every part is finite + pub const LANES_KEEP_OVERFLOW: u32 = 4; + /// number of ids + pub const N: u32 = 4; + /// the mutations' descriptions, by id + pub const NAMES: [&str; 5] = [ + "none", + "constructors: parts renormalized as given", + "constructors: no fallback", + "constructors: no retry after an overflow", + "constructors: lanes keep an overflow", + ]; + + #[cfg(feature = "mutate")] + thread_local!(static ID: std::cell::Cell = const { std::cell::Cell::new(0) }); + + /// select the mutation for the current thread (0 = none) + #[cfg(feature = "mutate")] + pub fn set(id: u32) { + ID.with(|m| m.set(id)); + } + + /// whether mutation id is selected in the current thread + #[cfg(feature = "mutate")] + #[inline(always)] + pub fn on(id: u32) -> bool { + ID.with(|m| m.get() == id) + } + + /// whether mutation id is selected: never, without the feature `mutate` + #[cfg(not(feature = "mutate"))] + #[inline(always)] + pub fn on(_id: u32) -> bool { + false + } +} + +/// x sorted by decreasing magnitude, stably: `f64::total_cmp` on `abs`, so that NaN parts come first and infinities +/// next. Parts already in that order are returned as they are, after N - 1 comparisons. +#[inline(always)] +pub(crate) fn by_magnitude(mut x: [f64; N]) -> [f64; N] { + if (0..N - 1).all(|k| x[k].abs() >= x[k + 1].abs()) { + return x; + } + for i in 1..N { + let mut j = i; + while j > 0 && x[j - 1].abs().total_cmp(&x[j].abs()) == Ordering::Less { + x.swap(j - 1, j); + j -= 1; + } + } + x +} + +/// ulp(a) exactly for finite a, 2^-1074 for zero and subnormal a, and infinity for infinite or NaN a: the power of two +/// of a's exponent field (at least 2^-1022) times 2^-52 +#[inline(always)] +pub(crate) fn ulp(a: f64) -> f64 { + f64::from_bits(a.to_bits() & 0x7ff0_0000_0000_0000).max(f64::MIN_POSITIVE) * f64::EPSILON +} + +/// x is on the ladder: zero parts only after the nonzero ones, and each part at most the ulp of the one before. A +/// quick test without branches, |x_(k+1)| 2^53 <= |x_k| for every k (exact, and so below the ulp), passes most parts +/// from the arithmetic; the others are compared with the ulp. +#[inline(always)] +pub(crate) fn on_ladder(x: &[f64; N]) -> bool { + let quick = (0..N - 1).fold(true, |m, k| m & (x[k + 1].abs() * 2f64.powi(53) <= x[k].abs())); + quick || (0..N - 1).all(|k| x[k + 1] == 0.0 || (x[k] != 0.0 && x[k + 1].abs() <= ulp(x[k]))) +} + +/// The exact sum of N finite parts, sorted by decreasing magnitude, as N parts on the ladder (see the module doc): +/// Grow-Expansion, then the pass from the largest component down. Each of the N - 1 steps of the pass emits at most one +/// part, so none is lost. A result that overflowed is formed again with each next part the largest left of the sign +/// opposite to the running sum. +#[cold] +#[inline(never)] +pub(crate) fn expansion_sum(x: [f64; N]) -> [f64; N] { + let r = grow_and_compress(x, false); + if r.iter().all(|p| p.is_finite()) || mutate::on(mutate::NO_BALANCE) { + r + } else { + grow_and_compress(x, true) + } +} + +/// `expansion_sum`'s steps, the parts added largest first or, `balanced`, each the largest left of the sign opposite +/// to the running sum +#[inline(always)] +fn grow_and_compress(x: [f64; N], balanced: bool) -> [f64; N] { + // e[..m] nonoverlapping, increasing in magnitude, zeros allowed; its last component is the running sum + let mut e = [0.0f64; N]; + let mut used = [false; N]; + for m in 0..N { + let next = if balanced { + let s = if m == 0 { 0.0 } else { e[m - 1] }; + (0..N) + .find(|&i| !used[i] && ((s > 0.0 && x[i] < 0.0) || (s < 0.0 && x[i] > 0.0))) + .or_else(|| (0..N).find(|&i| !used[i])) + .unwrap_or(m) + } else { + m + }; + used[next] = true; + let mut q = x[next]; + for c in e.iter_mut().take(m) { + let (s, h) = two_sum(q, *c); + *c = h; + q = s; + } + e[m] = q; + } + let mut out = [0.0f64; N]; + let mut k = 0; + let mut acc = e[N - 1]; + for i in (0..N - 1).rev() { + let (t, err) = two_sum(acc, e[i]); + if err != 0.0 { + out[k] = t; + k += 1; + acc = err; + } else { + acc = t; + } + } + out[k] = acc; + out +} + +/// The constructors' paths (see the module doc): `renorm`, the renormalization as the arithmetic runs it, for parts on +/// the ladder; otherwise the parts sorted and renormalized by `renorm_exact` (exact for N parts to N), kept when it is +/// finite and on the ladder or when a part is not finite, and `expansion_sum` of the sorted parts when not. +#[inline(always)] +pub(crate) fn exact_sum( + x: [f64; N], + renorm: impl FnOnce([f64; N]) -> [f64; N], + renorm_exact: impl FnOnce([f64; N]) -> [f64; N], +) -> [f64; N] { + if on_ladder(&x) { + return renorm(x); + } + let c = by_magnitude(x); + let r = renorm_exact(c); + let finite = c.iter().all(|p| p.is_finite()); + if !finite || (r[0].is_finite() && on_ladder(&r)) || mutate::on(mutate::NO_FALLBACK) { + r + } else { + expansion_sum(c) + } +} + +#[cfg(test)] +mod tests { + use crate::dd::{quick_two_sum, DD}; + use crate::exact::Dy; + use crate::od::{renorm_scalar, OD}; + use crate::qd::{renorm4, QD}; + + /// splitmix64 + struct Rng(u64); + + impl Rng { + fn next(&mut self) -> u64 { + self.0 = self.0.wrapping_add(0x9e37_79b9_7f4a_7c15); + let mut z = self.0; + z = (z ^ (z >> 30)).wrapping_mul(0xbf58_476d_1ce4_e5b9); + z = (z ^ (z >> 27)).wrapping_mul(0x94d0_49bb_1331_11eb); + z ^ (z >> 31) + } + /// a double-double with a random sign, an exponent in [-e, e] and a full low part + fn dd(&mut self, e: i32) -> DD { + let x = 2f64.powi((self.next() % (2 * e as u64 + 1)) as i32 - e) + * (1.0 + (self.next() >> 11) as f64 / 2f64.powi(53)); + let s = if self.next() & 1 == 0 { 1.0 } else { -1.0 }; + let lo = ((self.next() >> 11) as f64 / 2f64.powi(53) - 0.5) * x * f64::EPSILON; + DD::new(s * x, lo) + } + } + + /// the parts as bits + fn bits(x: [f64; N]) -> [u64; N] { + x.map(f64::to_bits) + } + + /// parts on the ladder: the first over the exponent range down to the subnormals, each next one a random fraction + /// of the ulp of the one before (the whole ulp one time in eight), signs at random + fn laddered(r: &mut Rng) -> [f64; N] { + let mut x = [0.0; N]; + x[0] = 2f64.powi((r.next() % 2023) as i32 - 1022) * (1.0 + (r.next() >> 11) as f64 / 2f64.powi(53)); + for k in 1..N { + let m = if r.next().is_multiple_of(8) { + 1.0 + } else { + (r.next() >> 11) as f64 / 2f64.powi(53) + }; + let s = if r.next() & 1 == 0 { 1.0 } else { -1.0 }; + x[k] = if x[k - 1] == 0.0 { + 0.0 + } else { + s * m * super::ulp(x[k - 1]) + }; + } + x + } + + /// r sums exactly to c and is on the ladder, by exact arithmetic + fn exact_on_ladder(c: &[f64], r: &[f64]) -> bool { + let ulp = |x: f64| { + let e = ((x.to_bits() >> 52) & 0x7ff) as i64; + Dy::from_f64(1.0).ldexp(if e == 0 { -1074 } else { e - 1075 }) + }; + Dy::sum(c).sub(&Dy::sum(r)).is_zero() + && (0..r.len() - 1).all(|k| { + if r[k] == 0.0 { + r[k + 1] == 0.0 + } else { + Dy::from_f64(r[k + 1]).cmp_abs(&ulp(r[k])) != std::cmp::Ordering::Greater + } + }) + } + + /// Parts on the ladder take the renormalization as given, whose quick two-sums are then exact and whose result is + /// on the ladder (the module doc's first path), checked by exact arithmetic. Control: the same parts reversed, + /// renormalized as given, fail the check. + #[test] + fn ladder_parts_renormalize_exactly_onto_the_ladder() { + let mut r = Rng(2); + let (mut reversed_fail, n) = (0, 20000); + for _ in 0..n { + let q: [f64; 4] = laddered(&mut r); + assert!(super::on_ladder(&q), "{q:?}"); + let rq = renorm4(q[0], q[1], q[2], q[3]); + assert!(exact_on_ladder(&q, &rq), "{q:?} gives {rq:?}"); + assert_eq!(bits(QD::new(q[0], q[1], q[2], q[3]).x), bits(rq)); + let o: [f64; 8] = laddered(&mut r); + assert!(super::on_ladder(&o), "{o:?}"); + let ro = renorm_scalar(o); + assert!(exact_on_ladder(&o, &ro), "{o:?} gives {ro:?}"); + assert_eq!(bits(OD::new(o).x), bits(ro)); + let p = [q[3], q[2], q[1], q[0]]; + reversed_fail += !exact_on_ladder(&p, &renorm4(p[0], p[1], p[2], p[3])) as usize; + } + println!("reversed parts: {reversed_fail} of {n} fail"); + assert!(reversed_fail > n / 10, "control: {reversed_fail} of {n}"); + } + + /// Parts from the arithmetic (largest first, on the ladder, zeros at the end) keep the bits of the constructors + /// before they took parts in any order: `renorm4`, `renorm_scalar` and the quick two-sum. Control: the same parts + /// permuted give other bits. + #[test] + fn ordered_parts_keep_their_bits() { + let mut r = Rng(1); + let (mut permuted_differ, n) = (0, 20000); + for i in 0..n { + let (a, b, c) = (r.dd(30), r.dd(30), r.dd(300)); + let d = match i % 3 { + 0 => a * b + c, + 1 => a / b, + _ => DD::new(a.hi, 0.0), + }; + // the two-sum's zero low part is +0, where the quick two-sum keeps a -0 low part + let (h, l) = quick_two_sum(d.hi, d.lo); + let e = DD::new(d.hi, d.lo); + assert_eq!(bits([e.hi, e.lo]), bits([h, l + 0.0]), "{d:?}"); + let q = match i % 3 { + 0 => QD::from_dd(a) * QD::from_dd(b) + QD::from_dd(c), + 1 => QD::from_dd(a) / QD::from_dd(b), + _ => QD::from_dd(a) + QD::from_dd(c), + }; + let x = q.x; + assert_eq!( + bits(QD::new(x[0], x[1], x[2], x[3]).x), + bits(renorm4(x[0], x[1], x[2], x[3])), + "{x:?}" + ); + let o = match i % 3 { + 0 => OD::from_qd(q) * OD::from_dd(c) + OD::from_dd(a), + 1 => OD::from_qd(q) / OD::from_dd(b), + _ => OD::from_qd(q) - OD::from_dd(a), + }; + assert_eq!(bits(OD::new(o.x).x), bits(renorm_scalar(o.x)), "{:?}", o.x); + let p = [o.x[3], o.x[0], o.x[6], o.x[1], o.x[7], o.x[2], o.x[5], o.x[4]]; + permuted_differ += (bits(OD::new(p).x) != bits(renorm_scalar(p))) as usize; + } + println!("permuted parts: {permuted_differ} of {n} differ from the renormalization as given"); + assert!(permuted_differ > n / 10, "control: {permuted_differ} of {n}"); + } +} diff --git a/src/dd.rs b/src/dd.rs index 043a475..08b7f09 100644 --- a/src/dd.rs +++ b/src/dd.rs @@ -80,9 +80,16 @@ impl DD { lo: 2.4492935982947064e-16, }; + /// hi + lo exactly, the two parts in either order: a two-sum, so `hi` is the sum rounded to nearest and `lo` its + /// error, at most ulp(hi)/2. Parts in that order already keep their bits, except that a zero low part is +0 (the + /// two-sum's error is never a negative zero). An infinite or NaN part gives a NaN low part. #[inline(always)] pub fn new(hi: f64, lo: f64) -> DD { - let (h, l) = quick_two_sum(hi, lo); + let (h, l) = if crate::any_order::mutate::on(crate::any_order::mutate::AS_GIVEN) { + quick_two_sum(hi, lo) + } else { + two_sum(hi, lo) + }; DD { hi: h, lo: l } } #[inline(always)] diff --git a/src/dv.rs b/src/dv.rs index 1416d59..83f2bd4 100644 --- a/src/dv.rs +++ b/src/dv.rs @@ -57,6 +57,8 @@ impl DV { lo: V8::zero(), } } + /// from the hi and lo arrays (8 values each), the parts as they are: |lo| at most ulp(hi)/2, as the arithmetic + /// returns them (`DD::new` takes them in either order) #[inline(always)] pub fn load(hi: &[f64], lo: &[f64]) -> Self { DV { diff --git a/src/lib.rs b/src/lib.rs index 8b67bb8..6028c69 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -19,6 +19,8 @@ //! fraction bits with the bits of exact integer arithmetic for |x_0| up to 2^52, and of their lanes [`dv::DV`], //! [`qv::QV`] and [`ov::OV`], each lane bitwise the scalar function of its value. //! * [`exact`]: exact dyadic and decimal arithmetic, the reference the tests compare against. +//! * [`any_order`]: how the constructors `QD::new`, `OD::new` and `OV::new` take their parts in any order, and the +//! controls' mutations of the constructors. //! * [`explog`]: `exp` and `ln` of [`dd::DD`], [`qd::QD`] and [`od::OD`], and of their lanes [`dv::DV`], [`qv::QV`] //! and [`ov::OV`] with each lane's bits. //! * [`precision`]: one trait, [`precision::Precision`], over double-double ([`precision::Double`]), quad-double @@ -66,6 +68,7 @@ #![allow(clippy::needless_range_loop, clippy::should_implement_trait)] #![cfg_attr(feature = "f128", feature(f128))] +pub mod any_order; pub mod dd; pub mod dual; pub mod dv; diff --git a/src/od.rs b/src/od.rs index e30ea03..c54fd6f 100644 --- a/src/od.rs +++ b/src/od.rs @@ -20,7 +20,11 @@ //! exact whatever the order of the first pass's output: a quick-two-sum is exact only when |acc| >= |c_i|, and a //! product of a value with a part far below the one above it (x0 + x1 with |x1| far below ulp(x0), or zero parts //! between nonzero ones) gives the first pass levels out of magnitude order, where a quick-two-sum dropped terms of -//! about 2^-325 (`od_tests`, category gap). The parts of such a result can still be out of magnitude order. +//! about 2^-325 (`od_tests`, category gap). The renormalization keeps the sum exact but does not sort, so parts out +//! of magnitude order can leave it out of order. `OD::new` sorts its parts first and checks its result +//! (`any_order`). The arithmetic's results are checked to stay on the ladder for operands on it +//! (`od_tests::od_results_do_not_overlap`), not for the gap category's operands (`gapped`), among them values with +//! zero parts between nonzero ones, which only a value built from raw parts can have. //! * Addition: the parts pairwise by two-sums, then each error carried down the levels below it, every level but //! the last by two-sums (`dep`), and the nine levels renormalized: QD's sloppy add with its error terms kept to //! the eighth level. Its error is relative to |a| + |b| (Cray-style), as QD's `+`. @@ -800,7 +804,8 @@ pub(crate) mod alg { /// Octo-double `x[0] + x[1] + ... + x[7]`. #[derive(Clone, Copy, Debug, Default, PartialEq)] pub struct OD { - /// the parts, largest first + /// the parts, largest first. `OD::new` puts parts given in any order so, each at most the ulp of the one before + /// and zeros only at the end; a value built from its fields must keep that order. pub x: [f64; 8], } @@ -814,10 +819,19 @@ impl OD { /// 2^-418, the unit of the error bounds (twice the exponent of QD's 2^-209; the parts hold 424 bits) pub const EPS: f64 = f64::from_bits((1023 - 418) << 52); - /// renormalized from eight parts + /// The exact sum of eight finite parts in any order, as eight parts largest first: zero parts only after the + /// nonzero ones, and each part at most the ulp of the one before (`any_order`). Parts already so, as the arithmetic + /// returns them, are renormalized as given (`renorm_scalar`, exact for eight parts) with its bits. Others are + /// sorted by decreasing magnitude and renormalized, and a result off the ladder (parts that overlap in one binade) + /// is replaced by the parts' exact expansion, compressed. With an infinite or NaN part the result is not finite. #[inline] pub fn new(x: [f64; 8]) -> OD { - OD { x: renorm_scalar(x) } + if crate::any_order::mutate::on(crate::any_order::mutate::AS_GIVEN) { + return OD { x: renorm_scalar(x) }; + } + OD { + x: crate::any_order::exact_sum(x, renorm_scalar, renorm_scalar), + } } /// exact #[inline(always)] diff --git a/src/ov.rs b/src/ov.rs index a60bf34..4f872e5 100644 --- a/src/ov.rs +++ b/src/ov.rs @@ -156,13 +156,49 @@ impl OV { pub fn zero() -> Self { OV { c: [V8::zero(); 8] } } - /// renormalized from eight parts, lane j bitwise `OD::new` of the parts' lane j + /// The exact sum of eight finite parts in any order on each lane, lane j bitwise `OD::new` of the parts' lane j. + /// When every lane's parts are in decreasing magnitude and every lane's renormalization comes out finite and on the + /// ladder (or has a part that is not finite), the lanes renormalize together (`OD::new` returns that renormalization + /// on such parts); otherwise each lane is `OD::new`. #[inline] pub fn new(c: [V8; 8]) -> Self { S::vectorize( #[inline(always)] - || OV { - c: as Limb>::renorm(c), + || { + use crate::any_order::mutate; + if mutate::on(mutate::AS_GIVEN) { + return OV { + c: as Limb>::renorm(c), + }; + } + let z = V8::::zero(); + let all = z.eq(z); + // NaN parts fail the comparison, and take OD::new's sort + let sorted = (0..7).fold(all, |m, k| m.and(c[k].abs().ge(c[k + 1].abs()))); + if sorted.bits() == all.bits() { + let r = as Limb>::renorm(c); + // as any_order::on_ladder, with any_order::ulp from the exponent bits + let (tiny, eps) = (V8::splat(f64::MIN_POSITIVE), V8::splat(f64::EPSILON)); + let ladder = (0..7).fold(all, |m, k| { + let ulp = r[k].pow2_floor().max(tiny).mul(eps); + m.and(r[k + 1].eq(z).or(r[k].ne(z).and(ulp.ge(r[k + 1].abs())))) + }); + // as any_order::exact_sum: a finite result on the ladder, or a part that is not finite + let inf = V8::splat(f64::INFINITY); + let finite = (0..8).fold(all, |m, k| m.and(inf.gt(c[k].abs()))); + let keep = inf.gt(r[0].abs()).and(ladder).or(finite.not()); + // the control keeps a first part that is not finite, also when every part is finite + let keep = if mutate::on(mutate::LANES_KEEP_OVERFLOW) { + keep.or(inf.gt(r[0].abs()).not()) + } else { + keep + }; + if keep.bits() == all.bits() || mutate::on(mutate::NO_FALLBACK) { + return OV { c: r }; + } + } + let parts = c.map(V8::to_array); + OV::from_lanes(&std::array::from_fn(|j| OD::new(std::array::from_fn(|k| parts[k][j])))) }, ) } @@ -232,7 +268,8 @@ impl OV { let (h, t) = alg::to_f64_parts(&self.c); t.eq(V8::zero()).sel(h, h.add(t)) } - /// from the eight parts' arrays (8 values each) + /// from the eight parts' arrays (8 values each), the parts as they are: largest first, as the arithmetic returns + /// them (`new` takes them in any order) #[inline(always)] pub fn load(c: [&[f64]; 8]) -> Self { let mut v = [V8::zero(); 8]; diff --git a/src/pack.rs b/src/pack.rs index e6cc5cd..7b39c6a 100644 --- a/src/pack.rs +++ b/src/pack.rs @@ -106,6 +106,7 @@ pub mod mutate { /// A number type stored as N `f64` parts, largest first. pub trait Parts: Copy { fn to_parts(self) -> [f64; N]; + /// the value of the parts as they are, largest first as `to_parts` gives them, not renormalized fn from_parts(p: [f64; N]) -> Self; /// whether the type is its N parts in order in memory and nothing else, so that the bulk paths may write it as /// `f64`: the crate's own types, checked again at run time (`f64s`) diff --git a/src/precision.rs b/src/precision.rs index 86f3eb4..08ad9de 100644 --- a/src/precision.rs +++ b/src/precision.rs @@ -61,7 +61,7 @@ pub trait Precision: Copy + Send + Sync + 'static { const ZERO: Self::Scalar; const ONE: Self::Scalar; - /// a scalar from up to PARTS parts, the missing ones zero, renormalized (`QD::new`, `OD::new`) + /// the exact sum of up to PARTS parts in any order, the missing ones zero (`DD::new`, `QD::new`, `OD::new`) /// /// # Panics /// `p` has more than PARTS parts. diff --git a/src/qd.rs b/src/qd.rs index 61b79ab..a5d9ab3 100644 --- a/src/qd.rs +++ b/src/qd.rs @@ -125,30 +125,54 @@ pub fn three_sum2(a: f64, b: f64, c: f64) -> (f64, f64) { /// QD's renormalization of four parts (no-op if c0 is infinite) #[inline(always)] pub fn renorm4(c0: f64, c1: f64, c2: f64, c3: f64) -> [f64; 4] { + renorm4_by(c0, c1, c2, c3, quick_two_sum) +} + +/// a + b as (s, e), error-free in any order: the quick two-sum where |a| >= |b|, which is then exact and gives the +/// two-sum's bits (up to the sign of a zero e), and the two-sum elsewhere +#[inline(always)] +fn exact_two_sum(a: f64, b: f64) -> (f64, f64) { + if a.abs() >= b.abs() { + quick_two_sum(a, b) + } else { + two_sum(a, b) + } +} + +/// `renorm4` with every step error-free whatever the order of the parts (`exact_two_sum`), so the four parts' sum is +/// kept exactly; bitwise `renorm4` wherever each of its quick two-sums is exact +#[inline(always)] +fn renorm4_exact(c: [f64; 4]) -> [f64; 4] { + renorm4_by(c[0], c[1], c[2], c[3], exact_two_sum) +} + +/// QD's renormalization of four parts with the two-sum `f` at every step +#[inline(always)] +fn renorm4_by(c0: f64, c1: f64, c2: f64, c3: f64, f: impl Fn(f64, f64) -> (f64, f64)) -> [f64; 4] { if c0.is_infinite() { return [c0, c1, c2, c3]; } - let (s, c3) = quick_two_sum(c2, c3); - let (s, c2) = quick_two_sum(c1, s); + let (s, c3) = f(c2, c3); + let (s, c2) = f(c1, s); let (c0, c1) = if mutate::on(mutate::RENORM_P1) { (c0 + s, 0.0) } else { - quick_two_sum(c0, s) + f(c0, s) }; let (mut s0, mut s1, mut s2, mut s3) = (c0, c1, 0.0, 0.0); if s1 != 0.0 { - (s1, s2) = quick_two_sum(s1, c2); + (s1, s2) = f(s1, c2); if s2 != 0.0 { - (s2, s3) = quick_two_sum(s2, c3); + (s2, s3) = f(s2, c3); } else { - (s1, s2) = quick_two_sum(s1, c3); + (s1, s2) = f(s1, c3); } } else { - (s0, s1) = quick_two_sum(s0, c2); + (s0, s1) = f(s0, c2); if s1 != 0.0 { - (s1, s2) = quick_two_sum(s1, c3); + (s1, s2) = f(s1, c3); } else { - (s0, s1) = quick_two_sum(s0, c3); + (s0, s1) = f(s0, c3); } } [s0, s1, s2, s3] @@ -274,7 +298,7 @@ fn quick_three_accum(a: &mut f64, b: &mut f64, c: f64) -> f64 { /// Quad-double `x[0] + x[1] + x[2] + x[3]`. The arithmetic expects parts as its own results, `new` and `from_sum` give /// them: decreasing in magnitude, with zeros only at the end. Parts with a zero between nonzero ones, as a `QD { x }` /// built by hand can have, can make a sum lose terms of about 2^-160 relative (`qd_tests::qd_noncanonical_add_control`); -/// `from_sum` renormalizes parts in any order. +/// `new` and `from_sum` take parts in any order. #[derive(Clone, Copy, Debug, Default, PartialEq)] pub struct QD { pub x: [f64; 4], @@ -306,11 +330,20 @@ impl QD { /// QD's epsilon, 2^-209 (exactly; QD's decimal literal 1.21543267145725e-63 is its 15-digit rounding) pub const EPS: f64 = f64::from_bits(0x32e0_0000_0000_0000); - /// renormalized from four parts + /// The exact sum of four finite parts in any order, as four parts largest first: zero parts only after the + /// nonzero ones, and each part at most the ulp of the one before (`any_order`). Parts already so, as the arithmetic + /// returns them, are renormalized by `renorm4` with its bits. Others are sorted by decreasing magnitude and + /// renormalized with error-free steps, and a result off the ladder (parts that overlap in one binade) is replaced + /// by the parts' exact expansion, compressed. With an infinite or NaN part the result is not finite. #[inline(always)] pub fn new(x0: f64, x1: f64, x2: f64, x3: f64) -> QD { + if crate::any_order::mutate::on(crate::any_order::mutate::AS_GIVEN) { + return QD { + x: renorm4(x0, x1, x2, x3), + }; + } QD { - x: renorm4(x0, x1, x2, x3), + x: crate::any_order::exact_sum([x0, x1, x2, x3], |c| renorm4(c[0], c[1], c[2], c[3]), renorm4_exact), } } /// the exact sum of four parts in any order, renormalized (`renorm4_any`); bitwise `new` on normalized parts diff --git a/src/qv.rs b/src/qv.rs index 06cab7b..3687583 100644 --- a/src/qv.rs +++ b/src/qv.rs @@ -246,7 +246,8 @@ impl QV { let t = self.c[1].add(self.c[2].add(self.c[3])); t.eq(V8::zero()).sel(self.c[0], self.c[0].add(t)) } - /// from the four parts' arrays (8 values each) + /// from the four parts' arrays (8 values each), the parts as they are: largest first, as the arithmetic returns + /// them (`QD::new` takes them in any order) #[inline(always)] pub fn load(c0: &[f64], c1: &[f64], c2: &[f64], c3: &[f64]) -> Self { QV { diff --git a/tests/any_order.rs b/tests/any_order.rs new file mode 100644 index 0000000..a55d968 --- /dev/null +++ b/tests/any_order.rs @@ -0,0 +1,458 @@ +//! The constructors `DD::new`, `QD::new`, `OD::new` and `OV::new` on parts in any order. Each result is the exact sum +//! of the parts (`exact::Dy`), with zero parts only after the nonzero ones and each part at most the ulp of the one +//! before, for parts in random order with cancelling pairs, zeros between nonzero parts, huge and tiny parts, signed +//! zeros, subnormals and parts that overlap in one binade. Parts with a NaN or an infinity give a first part that is +//! not finite, and the lanes give the scalar bits on every backend. Controls (feature `mutate`): the constructors +//! as they were (`any_order::mutate::AS_GIVEN`), without their fallback (`NO_FALLBACK`), without the fallback's retry +//! after an overflow (`NO_BALANCE`), and lanes that keep an overflow (`LANES_KEEP_OVERFLOW`) fail these checks. +mod common; +use common::Rng; +use multidouble::dd::DD; +use multidouble::exact::Dy; +use multidouble::od::OD; +use multidouble::ov::OV; +use multidouble::qd::QD; +use multidouble::simd::{self, Kernel, Simd}; +use multidouble::v8::V8; +use std::cmp::Ordering; + +/// cases per type, fewer in the unoptimized coverage build +fn cases() -> usize { + if cfg!(debug_assertions) { + 20000 + } else { + 60000 + } +} + +/// the unit in the last place of a finite x (the subnormal spacing for zero and subnormal x), exactly +fn ulp(x: f64) -> Dy { + let e = ((x.to_bits() >> 52) & 0x7ff) as i64; + Dy::from_f64(1.0).ldexp(if e == 0 { -1074 } else { e - 1075 }) +} + +/// how r fails the constructors' contract for the parts c: "value" if its sum is not c's, "ladder" if a zero part +/// precedes a nonzero one or a part exceeds the ulp of the one before +fn faults(c: &[f64], r: &[f64]) -> (bool, bool) { + // a part that is not finite (a partial sum that overflowed) is a wrong value + if !r.iter().all(|x| x.is_finite()) { + return (true, false); + } + let value = !Dy::sum(c).sub(&Dy::sum(r)).is_zero(); + let ladder = (0..r.len() - 1).any(|k| { + if r[k] == 0.0 { + r[k + 1] != 0.0 + } else { + Dy::from_f64(r[k + 1]).cmp_abs(&ulp(r[k])) == Ordering::Greater + } + }); + (value, ladder) +} + +/// the largest |r_(k+1)| / ulp(r_k) over r's nonzero parts +fn worst_ratio(r: &[f64]) -> f64 { + (0..r.len() - 1) + .filter(|&k| r[k] != 0.0) + .map(|k| Dy::from_f64(r[k + 1]).ratio(&ulp(r[k]))) + .fold(0.0, f64::max) +} + +/// +1 or -1 with equal odds +fn sign(r: &mut Rng) -> f64 { + if r.next_u64() & 1 == 0 { + 1.0 + } else { + -1.0 + } +} + +/// an integer in [lo, hi] +fn int(r: &mut Rng, lo: i32, hi: i32) -> i32 { + lo + (r.next_u64() % (hi - lo + 1) as u64) as i32 +} + +/// a double with a random sign and an exponent in [lo, hi], a zero of either sign one time in eight +fn double(r: &mut Rng, lo: i32, hi: i32) -> f64 { + if r.next_u64().is_multiple_of(8) { + return 0.0 * sign(r); + } + (1.0 + r.unit()) * 2f64.powi(int(r, lo, hi)) * sign(r) +} + +/// m 2^e with m in 1..15 and e in [lo, hi], zero one time in six: few significant bits, so that sums are often exact +/// and ties common +fn sparse(r: &mut Rng, lo: i32, hi: i32) -> f64 { + if r.next_u64().is_multiple_of(6) { + return 0.0; + } + (1 + r.next_u64() % 15) as f64 * 2f64.powi(int(r, lo, hi)) * sign(r) +} + +/// x in a random order (Fisher-Yates) +fn permuted(r: &mut Rng, mut x: [f64; N]) -> [f64; N] { + for i in (1..N).rev() { + x.swap(i, (r.next_u64() % (i as u64 + 1)) as usize); + } + x +} + +/// the parts of a value from the N-part arithmetic, largest first +fn normalized(r: &mut Rng) -> [f64; N] { + let (a, b, c) = (r.dd(30), r.dd(30), r.dd(30)); + let p: Vec = match N { + 2 => vec![(a * b + c).hi, (a * b + c).lo], + 4 => (QD::from_dd(a) * QD::from_dd(b) + QD::from_dd(c)).x.to_vec(), + _ => (OD::from_qd(QD::from_dd(a) * QD::from_dd(b)) * OD::from_dd(c) + OD::from_dd(r.dd(30))) + .x + .to_vec(), + }; + std::array::from_fn(|i| p[i]) +} + +/// the families of inputs, by i +const FAMILIES: usize = 12; + +/// the ulp of a finite x, 2^-1074 for zero and subnormal x +fn ulp_f64(x: f64) -> f64 { + f64::from_bits(x.to_bits() & 0x7ff0_0000_0000_0000).max(f64::MIN_POSITIVE) * f64::EPSILON +} + +/// parts of family i % FAMILIES: a normalized value's parts permuted; cancelling pairs among other parts; +/// independent parts over 240 binades; parts in four nearby binades; few-bit parts over 60 binades; few-bit parts in +/// seven binades; a huge cancelling pair with tiny parts (the shape of the report on PR #89); parts over the whole +/// exponent range with subnormals; positive parts in one binade (whose sorted renormalization can come out off the +/// ladder); signed zeros among few-bit parts; parts already on the ladder, each up to a whole ulp of the one before +/// and the first down to the subnormals (the constructors' path that renormalizes them as given); two parts of one +/// sign, cancelled by two smaller ones of the other sign whose sum overflows, among smaller parts +fn inputs(r: &mut Rng, i: usize) -> [f64; N] { + match i % FAMILIES { + 0 => { + let x = normalized(r); + permuted(r, x) + } + 1 => { + let mut x: [f64; N] = std::array::from_fn(|_| double(r, -60, 60)); + for p in 0..1 + (r.next_u64() % (N as u64 / 2)) as usize { + let y = x[2 * p]; + x[2 * p + 1] = match r.next_u64() % 3 { + 0 => -y, + 1 => -y * (1.0 + f64::EPSILON * (r.next_u64() % 8) as f64), + _ => y * 2f64.powi(-int(r, 0, 200)), + }; + } + permuted(r, x) + } + 2 => std::array::from_fn(|_| double(r, -120, 120)), + 3 => { + let e = int(r, -20, 20); + std::array::from_fn(|_| double(r, e, e + 3)) + } + 4 => { + let e = int(r, -20, 20); + std::array::from_fn(|_| sparse(r, e - 60, e + 3)) + } + 5 => { + let e = int(r, -5, 5); + std::array::from_fn(|_| sparse(r, e - 3, e + 3)) + } + 6 => { + let mut x: [f64; N] = std::array::from_fn(|_| sparse(r, -300, 2)); + let h = sparse(r, 40, 120); + (x[0], x[1]) = (h, -h); + permuted(r, x) + } + 7 => { + // below 2^1018, so that no partial sum of eight parts overflows + let mut x: [f64; N] = std::array::from_fn(|_| double(r, -1074, 1018)); + x[0] = f64::from_bits(r.next_u64() % (1 << 52)) * sign(r); + permuted(r, x) + } + 8 => { + let e = int(r, -4, 4); + std::array::from_fn(|_| (1.0 + r.unit()) * 2f64.powi(e)) + } + 9 => std::array::from_fn(|_| { + if r.next_u64() & 1 == 0 { + 0.0 * sign(r) + } else { + sparse(r, -10, 10) + } + }), + 11 => { + // a, b in [1.5, 1.75) 2^1023 and c, d in [1.25, 1.5) 2^1023 of the other sign: sorted, c + d overflows + // first, and the sum lies in (-2^1023, 2^1023) + let mut x: [f64; N] = std::array::from_fn(|_| double(r, -100, 1000)); + let s = sign(r); + let mut big = |lo: f64| (lo + 0.25 * r.unit()) * 2f64.powi(1023); + let (a, b, c, d) = (big(1.5) * s, big(1.5) * s, -big(1.25) * s, -big(1.25) * s); + if N >= 4 { + (x[0], x[1], x[2], x[3]) = (a, b, c, d); + } else { + (x[0], x[1]) = (a, c); + } + permuted(r, x) + } + _ => { + let mut x = [0.0; N]; + x[0] = (1.0 + r.unit()) * 2f64.powi(int(r, -1022, 1000)) * sign(r); + // the parts after a random one zeroed one time in four + let end = if r.next_u64().is_multiple_of(4) { + 1 + (r.next_u64() % N as u64) as usize + } else { + N + }; + for k in 1..end { + let u = ulp_f64(x[k - 1]); + let m = if r.next_u64().is_multiple_of(8) { 1.0 } else { r.unit() }; + x[k] = if x[k - 1] == 0.0 { 0.0 } else { m * u * sign(r) }; + } + x + } + } +} + +/// the scalar constructor of N parts +fn construct(x: [f64; N]) -> [f64; N] { + let p: Vec = match N { + 2 => { + let d = DD::new(x[0], x[1]); + vec![d.hi, d.lo] + } + 4 => QD::new(x[0], x[1], x[2], x[3]).x.to_vec(), + _ => OD::new(std::array::from_fn(|i| x[i])).x.to_vec(), + }; + std::array::from_fn(|i| p[i]) +} + +/// a constructor's faults on `cases()` unordered inputs +struct Tally { + /// results whose sum is not the parts' + value: usize, + /// results off the ladder + ladder: usize, + /// the largest |x_(k+1)| / ulp(x_k) of the results + worst: f64, + /// the parts of a faulty case + example: Option>, +} + +/// The constructor of N parts on `cases()` unordered inputs from the seed +fn run(seed: u64) -> Tally { + let mut r = Rng(seed); + let mut t = Tally { + value: 0, + ladder: 0, + worst: 0.0, + example: None, + }; + for i in 0..cases() { + let c: [f64; N] = inputs(&mut r, i); + let x = construct(c); + let (v, l) = faults(&c, &x); + t.value += v as usize; + t.ladder += l as usize; + if (v || l) && t.example.is_none() { + t.example = Some(c.to_vec()); + } + if x.iter().all(|p| p.is_finite()) { + t.worst = t.worst.max(worst_ratio(&x)); + } + } + t +} + +/// the three constructors' tallies, on the same inputs whatever the mutation +fn tallies() -> [(&'static str, Tally); 3] { + [ + ("DD::new", run::<2>(1)), + ("QD::new", run::<4>(2)), + ("OD::new", run::<8>(3)), + ] +} + +/// the constructors' results are the exact sums of the parts, on the ladder, for every family +#[test] +fn constructors_sum_exactly_onto_the_ladder() { + for (name, t) in tallies() { + println!("{name}: {} cases, worst |x_(k+1)| / ulp(x_k) {:.4}", cases(), t.worst); + assert!( + t.value == 0 && t.ladder == 0, + "{name}: {} wrong values, {} off the ladder, e.g. {:?}", + t.value, + t.ladder, + t.example + ); + } +} + +/// A NaN or an infinity among the parts, in any position, gives a first part that is not finite. Control: the same +/// parts with a zero in its place give a finite one. +#[test] +fn non_finite_parts_give_a_non_finite_result() { + let mut r = Rng(4); + let specials = [f64::NAN, f64::INFINITY, f64::NEG_INFINITY]; + let not_finite = |x: &[f64]| !x[0].is_finite(); + // the overflow family's parts, one replaced by zero, can sum past the largest double + for i in (0..300).filter(|i| i % FAMILIES != 11) { + let s = specials[i % 3]; + let mut c2: [f64; 2] = inputs(&mut r, i); + let mut c4: [f64; 4] = inputs(&mut r, i); + let mut c8: [f64; 8] = inputs(&mut r, i); + let (k2, k4, k8) = ( + (r.next_u64() % 2) as usize, + (r.next_u64() % 4) as usize, + (r.next_u64() % 8) as usize, + ); + (c2[k2], c4[k4], c8[k8]) = (0.0, 0.0, 0.0); + for (c, x) in [ + (c2.to_vec(), construct(c2).to_vec()), + (c4.to_vec(), construct(c4).to_vec()), + (c8.to_vec(), construct(c8).to_vec()), + ] { + assert!(!not_finite(&x), "control: {c:?} gives {x:?}"); + } + (c2[k2], c4[k4], c8[k8]) = (s, s, s); + for (c, x) in [ + (c2.to_vec(), construct(c2).to_vec()), + (c4.to_vec(), construct(c4).to_vec()), + (c8.to_vec(), construct(c8).to_vec()), + ] { + assert!(not_finite(&x), "{c:?} gives {x:?}"); + } + } +} + +/// The report of PR #89 and the two like it: `DD::new(2^-60, 1)` gave 1, `QD::new(2^-213, 2^54, 1, -2^54)` gave +/// 2^-213, and `OD::new` of the same with 2^-270 gave the parts [2^-213, 1, 2^-270], whose reciprocal was NaN and whose +/// cosine was 1. Parts near the overflow threshold whose sorted sum overflows give their finite sum. +#[test] +fn reported_cases() { + let p = |e: i32| 2f64.powi(e); + let d = DD::new(p(-60), 1.0); + assert_eq!((d.hi, d.lo), (1.0, p(-60))); + let q = QD::new(p(-213), p(54), 1.0, -p(54)); + assert_eq!(q.x, [1.0, p(-213), 0.0, 0.0]); + let o = OD::new([p(-213), p(54), p(-270), 1.0, -p(54), 0.0, 0.0, 0.0]); + assert_eq!(o.x, [1.0, p(-213), p(-270), 0.0, 0.0, 0.0, 0.0, 0.0]); + let inv = OD::ONE / o; + assert!(inv.x.iter().all(|x| x.is_finite()) && inv.to_f64() == 1.0, "{inv:?}"); + // cos 1 = 0.5403023058681398 to f64; the tiny-argument shortcut would give 1 + let c = o.cos().to_f64(); + assert!((c - 1f64.cos()).abs() < 1e-15, "{c}"); + // Greptile's case on this PR: sorted, (-h, -h, p, p) sums p + p first, which overflows, where the sum is -p + let h = 1.5 * p(1023); + assert_eq!(QD::new(-h, p(1023), -h, p(1023)).x, [-p(1023), 0.0, 0.0, 0.0]); + let o = OD::new([-h, p(1023), -h, p(1023), 0.0, 0.0, 0.0, 0.0]); + assert_eq!(o.x, [-p(1023), 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]); + // CodeRabbit's case: sorted, (-M, 2^1023, 2^1023) sums 2^1023 + 2^1023 first, which overflows, where the sum is + // 2^971 + let m = f64::MAX; + assert_eq!(QD::new(p(1023), p(1023), -m, 0.0).x, [p(971), 0.0, 0.0, 0.0]); + let o = OD::new([p(1023), p(1023), -m, 0.0, 0.0, 0.0, 0.0, 0.0]); + assert_eq!(o.x, [p(971), 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]); +} + +/// `OV::new` against `OD::new` lane by lane on one backend: the lanes whose parts differ in any bit (NaNs alike) +#[derive(Clone)] +struct OvNewMatchesOdNew; + +impl Kernel for OvNewMatchesOdNew { + type Output = usize; + /// rounds of 8 lanes: all normalized (the lanes renormalize together), all sorted and in one binade or in the + /// overflow family (some or all lanes take the fallback), every family mixed, and a NaN or an infinity on one lane + #[inline(always)] + fn run(self) -> usize { + let mut r = Rng(5); + let mut bad = 0; + for round in 0..cases() / 8 { + let x: [[f64; 8]; 8] = std::array::from_fn(|j| match round % 4 { + 0 => normalized(&mut r), + 1 => { + // the overflow family's sorted parts overflow in the lanes' renormalization + let mut c: [f64; 8] = inputs(&mut r, if round % 8 == 1 { 8 } else { 11 }); + c.sort_by(|a, b| b.abs().total_cmp(&a.abs())); + c + } + 2 => inputs(&mut r, round + j), + _ => { + let mut c: [f64; 8] = normalized(&mut r); + if j == round / 4 % 8 { + c[(r.next_u64() % 8) as usize] = [f64::NAN, f64::INFINITY, f64::NEG_INFINITY][round / 32 % 3]; + } + c + } + }); + let v = OV::::new(std::array::from_fn(|k| { + V8::from_array(std::array::from_fn(|j| x[j][k])) + })); + for (j, &xj) in x.iter().enumerate() { + let (g, w) = (v.lane(j), OD::new(xj)); + let same = + g.x.iter() + .zip(w.x) + .all(|(a, b)| a.to_bits() == b.to_bits() || (a.is_nan() && b.is_nan())); + bad += !same as usize; + } + } + bad + } +} + +/// `OV::new` gives `OD::new`'s bits on every lane, on every backend this CPU runs +#[test] +fn ov_new_matches_od_new_in_any_order() { + let ran = simd::for_each_available(OvNewMatchesOdNew); + assert_eq!(ran.iter().map(|r| r.0).collect::>(), simd::available()); + assert!(ran.iter().all(|r| r.1 == 0), "lanes that differ: {ran:?}"); + println!("checked on {:?}", simd::available()); +} + +/// The controls: the constructors as they were fail the checks (`DD::new` and `QD::new` on the value, `QD::new` and +/// `OD::new` on the ladder), without the fallback `QD::new` and `OD::new` leave parts off the ladder and overflow on the +/// overflow family, and without the fallback's retry after an overflow they overflow there. The lanes follow the +/// scalar constructor under each. +#[cfg(feature = "mutate")] +#[test] +fn controls_fail_the_checks() { + use multidouble::any_order::mutate; + for id in [mutate::AS_GIVEN, mutate::NO_FALLBACK, mutate::NO_BALANCE] { + mutate::set(id); + let rows = tallies(); + let lanes = simd::for_each_available(OvNewMatchesOdNew); + mutate::set(0); + for (name, t) in &rows { + println!( + "'{}': {name}: {} wrong values, {} off the ladder, worst |x_(k+1)| / ulp(x_k) {:.4}", + mutate::NAMES[id as usize], + t.value, + t.ladder, + t.worst + ); + } + assert!(lanes.iter().all(|r| r.1 == 0), "lanes that differ: {lanes:?}"); + let [dd, qd, od] = rows.map(|(_, t)| (t.value > 0, t.ladder > 0)); + if id == mutate::AS_GIVEN { + assert!(dd.0 && qd.0 && qd.1 && od.1, "{dd:?} {qd:?} {od:?}"); + } else if id == mutate::NO_FALLBACK { + assert!(qd.0 && qd.1 && od.0 && od.1, "{qd:?} {od:?}"); + } else { + assert!(qd.0 && od.0, "{qd:?} {od:?}"); + } + } +} + +/// The lanes' control: `OV::new` keeping the lanes' renormalization when its first part is not finite differs from +/// `OD::new` on the sorted lanes of the overflow family, on every backend this CPU runs +#[cfg(feature = "mutate")] +#[test] +fn lanes_control_fails_the_check() { + use multidouble::any_order::mutate; + mutate::set(mutate::LANES_KEEP_OVERFLOW); + let lanes = simd::for_each_available(OvNewMatchesOdNew); + mutate::set(0); + println!( + "'{}': lanes that differ: {lanes:?}", + mutate::NAMES[mutate::LANES_KEEP_OVERFLOW as usize] + ); + assert!(lanes.iter().all(|r| r.1 > 0), "lanes that differ: {lanes:?}"); +} diff --git a/tests/from_sum.rs b/tests/from_sum.rs index 604f07d..caaf648 100644 --- a/tests/from_sum.rs +++ b/tests/from_sum.rs @@ -1,12 +1,12 @@ //! `QD::from_sum` and `qv::renorm4_any`: four parts given in any order, summed exactly and renormalized. Checked //! against exact dyadic arithmetic (`exact::Dy`), bitwise `QD::new` on normalized parts, and the 8 lanes bitwise the -//! scalar on every backend this CPU runs. Control: `QD::new` itself, which assumes ordered parts, must fail the -//! exactness check on unordered ones. +//! scalar on every backend this CPU runs. Control: `renorm4`, which assumes ordered parts (and was `QD::new` before it +//! took parts in any order), must fail the exactness check on unordered ones. mod common; use common::Rng; use multidouble::dd::DD; use multidouble::exact::Dy; -use multidouble::qd::{renorm4_any, QD}; +use multidouble::qd::{renorm4, renorm4_any, QD}; use multidouble::qv; use multidouble::simd::{self, Kernel, Simd}; use multidouble::v8::V8; @@ -139,18 +139,18 @@ fn from_sum_is_new_on_normalized_parts() { } } -/// the check can fail: `QD::new` gives 1 for CANCEL, and misses on a share of the other unordered parts +/// the check can fail: `renorm4` gives 1 for CANCEL, and misses on a share of the other unordered parts #[test] -fn new_fails_the_check_on_unordered_parts() { - let z = QD::new(CANCEL[0], CANCEL[1], CANCEL[2], CANCEL[3]); - assert!(!exact_and_normalized(CANCEL, z.x), "{:?}", z.x); +fn renorm4_fails_the_check_on_unordered_parts() { + let z = renorm4(CANCEL[0], CANCEL[1], CANCEL[2], CANCEL[3]); + assert!(!exact_and_normalized(CANCEL, z), "{z:?}"); let mut r = Rng(17); let missed = (0..N) .map(|i| unordered(&mut r, i)) - .filter(|&c| !exact_and_normalized(c, QD::new(c[0], c[1], c[2], c[3]).x)) + .filter(|&c| !exact_and_normalized(c, renorm4(c[0], c[1], c[2], c[3]))) .count(); - println!("QD::new misses {missed} of {N} unordered cases"); - assert!(missed >= N / 10, "QD::new missed only {missed} of {N}"); + println!("renorm4 misses {missed} of {N} unordered cases"); + assert!(missed >= N / 10, "renorm4 missed only {missed} of {N}"); } /// `qv::renorm4_any` on 8 lanes of mixed normalized, unordered and non-finite parts against the scalar, bit for bit