Skip to content

fix: keep selected locations when supplementing - #23

Open
ChrisW09 wants to merge 1 commit into
mainfrom
fix/supplement-keeps-selected-locations
Open

fix: keep selected locations when supplementing#23
ChrisW09 wants to merge 1 commit into
mainfrom
fix/supplement-keeps-selected-locations

Conversation

@ChrisW09

@ChrisW09 ChrisW09 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Fixes #18.

Problem

When a target-aware selector returns fewer locations than min_count, _supplement tops
the set up with quantile candidates:

quantile_candidates = quantile_knots(x, target_count)
all_locations = set(existing) | set(quantile_candidates.tolist())
return sorted(all_locations)[:target_count]

quantile_knots already returns target_count entries, so the union always overflows by
exactly len(existing) — and truncating a sorted list always removes the largest values.
The step meant to add to the selector's findings systematically deleted them instead,
whenever they sat above the median.

Fix

Keep every existing location; fill only the shortfall. The quantile candidates used as
filler are down-sampled with select_knots (even spacing) rather than sliced, so the
additions stay spread across the range instead of bunching at the low end.

Result

Isolating the call, with three tree-selected locations and a target of 6:

existing : [-0.995, -0.026,  0.969]
before   : [-0.995 -0.714 -0.429 -0.143 -0.026  0.143]   <- 2 of 3 kept, all output <= 0.143
after    : [-0.995 -0.714 -0.143 -0.026  0.714  0.969]   <- 3 of 3 kept, spans the range

End to end, a feature with few distinct values (so the tree places only three spaced splits
against a min_count of 6) now retains all three:

rng = np.random.default_rng(0)
xs = rng.choice([0.0, 0.05, 9.5, 9.8], size=300)
ys = (xs > 5).astype(float) + 0.01 * rng.normal(size=300)
Preprocessor(numerical_method="ple", output_dim=7).fit(pd.DataFrame({"x": xs}), ys)
# thresholds: [-1. -0.995 -0.99 -0.026 0.969 1.]

A correction to the issue

The second reproduction in #18's description did not actually reach _supplement — I
instrumented it and the method is never called for that input, so the bunching it showed has
a different cause. I have commented on the issue
with that correction and a repro that does exercise the path. The first reproduction in the
issue was accurate.

Tests

Four added to tests/test_location_selectors.py:

  • test_supplement_keeps_every_existing_location — the reported case
  • test_supplement_fills_only_the_shortfall
  • test_supplement_is_a_noop_when_already_full
  • test_supplement_spreads_the_added_candidates — guards the low-end bias specifically

Full suite: 484 passed, 9 xfailed. ruff check clean on both changed files; pyright
unchanged at its pre-existing 71 errors.

🤖 Generated with Claude Code

``_supplement`` merged the selector's locations with a full set of quantile
candidates and then truncated the union with ``sorted(...)[:target_count]``.
``quantile_knots`` already returns ``target_count`` entries, so the union
always overflowed by exactly ``len(existing)`` and the truncation reliably
discarded the largest values -- the data-driven locations the selector had
just found, whenever they sat above the median.

Keep every existing location and fill only the shortfall, down-sampling the
quantile candidates by even spacing so the additions stay spread across the
range rather than bunching at the low end.

    existing : [-0.995, -0.026,  0.969]
    before   : [-0.995 -0.714 -0.429 -0.143 -0.026  0.143]   2 of 3 kept
    after    : [-0.995 -0.714 -0.143 -0.026  0.714  0.969]   3 of 3 kept

Closes #18

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ChrisW09
ChrisW09 force-pushed the fix/supplement-keeps-selected-locations branch from 66817f8 to 25cf672 Compare July 27, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: location supplement drops tree-selected high-end locations

1 participant