fix(splines): stop adding a collinear bias column to the B-spline basis - #39
Open
ChrisW09 wants to merge 1 commit into
Open
fix(splines): stop adding a collinear bias column to the B-spline basis#39ChrisW09 wants to merge 1 commit into
ChrisW09 wants to merge 1 commit into
Conversation
``BSplineTransformer`` defaulted to ``include_bias=True`` and was the only
spline family that did. A B-spline basis over a clamped knot vector is a
partition of unity -- every row sums to exactly 1 -- so the prepended intercept
column is an exact linear combination of the remaining columns and the design
matrix is singular:
include_bias=True (200, 9) rank 8 cond 6.30e+15
include_bias=False (200, 8) rank 8 cond 1.79e+01
``Preprocessor(numerical_method="bspline")`` inherited that default, so every
user of the method got a rank-deficient design: an unidentifiable intercept for
OLS, and a wasted direction for a regularized fit.
Default ``include_bias`` to False, matching MSpline, ISpline and the shared
base class. ``include_bias=True`` remains available for callers that want the
explicit column.
The same change resolves two further symptoms of the same cause. bspline was
the only method whose per-feature width differed from ``output_dim`` (8 for
output_dim=7, where every other family gave 7), and the only one that could
breach the adaptive window (10 against max_output_dim=9). Both now match.
``test_fixed_width_matches_expected[bspline]`` encoded the old ``output_dim + 1``
width and is updated; the two docstring examples showing ``(50, 9)`` are too.
Closes #33
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #33.
Problem
BSplineTransformerdefaulted toinclude_bias=True, and it was the only spline family thatdid. A B-spline basis over a clamped knot vector is a partition of unity — every row sums
to exactly 1 — so the prepended intercept column is an exact linear combination of the rest:
Preprocessor(numerical_method="bspline")inherited that default, so this was theout-of-the-box path, not an opt-in. Downstream, OLS has an unidentifiable intercept and a
regularized fit spends a coefficient on a direction that carries no information.
Fix
Default
include_biastoFalse, matchingMSplineTransformer,ISplineTransformerandBaseSplineTransformer.include_bias=Truestays available for callers who want the columnand accept the collinearity, and the docstring now explains why it is off by default.
One cause, three symptoms
The same change fixes two things filed alongside it in the issue:
matrix_rank/ columnsoutput_dims_atoutput_dim=7output_dims_atadaptive,max_output_dim=9The width mismatch mattered beyond aesthetics:
include_biasis not aPreprocessorparameter, so there was no way to opt out, and
max_output_dim— which users set to boundwidth — was silently exceeded.
Tests
Four added to
tests/test_spline_expansions.py:test_bspline_default_design_is_full_rank— rank andcond < 1e6; fails on the old defaulttest_bspline_basis_is_a_partition_of_unity— pins why the bias column is redundant, sothe reasoning is checked rather than just asserted in a comment
test_bspline_include_bias_still_available_opt_intest_bmi_splines_default_to_exactly_output_dim, parametrised over B/M/IChanged existing expectations
test_fixed_width_matches_expected[bspline]pinnedOUTPUT_DIM + 1with the comment"B-spline defaults to include_bias=True -> output_dim + 1" — i.e. it encoded the asymmetry
being fixed. Updated to
OUTPUT_DIM. Two docstring examples showing(50, 9)are now(50, 8).This changes a documented default. I think it is right — the old default produces a
singular matrix, which is a defect rather than a preference — but it is a behaviour change,
so flagging it explicitly. If you would rather keep the default and only warn when
include_bias=Trueis requested on a partition-of-unity basis, say so and I will swap it.Related, not changed here:
MSplineTransformer(include_bias=True)is also rank-deficient(M-splines are rescaled B-splines, so their span also contains the constants —
rank 8/9).That one is opt-in, so I left it alone; a warning would be a reasonable follow-up.
ISplineTransformer(include_bias=True)is full rank and unaffected.Full suite: 486 passed, 9 xfailed; 24 doctests pass.
ruff checkclean on changed files;pyright unchanged at its pre-existing 71 errors.
🤖 Generated with Claude Code