fix(onehot): encode categories before onehot_from_ordinal - #28
Open
ChrisW09 wants to merge 1 commit into
Open
Conversation
``categorical_method="onehot_from_ordinal"`` appended only
``OneHotFromOrdinalTransformer``, which requires input that is already ordinal
encoded. The Preprocessor hands it raw column values, so any string column died
inside ``np.max(X, axis=0).astype(int)`` with a bare
ValueError: invalid literal for int() with base 10: 'c'
that says nothing about the chosen method. Since ``_detect_column_types`` routes
every non-numeric column to the categorical side, this was the common case.
Insert ``ContinuousOrdinalTransformer`` ahead of it so the method does what the
Preprocessor docstring already claimed -- integer codes, then one-hot. Because
that encoder numbers categories from 1 and reserves 0 for unseen values, a
column with k categories expands to k + 1 columns and categories unseen at fit
time land in the reserved column instead of producing an all-zero row.
Docs reconciled: the Preprocessor docstring, the user-guide table and the class
itself previously gave three different accounts of this method.
Closes #17
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChrisW09
force-pushed
the
fix/onehot-from-ordinal-encodes-strings
branch
from
July 27, 2026 20:26
ef00389 to
0cb45b0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #17.
Problem
categorical_method="onehot_from_ordinal"appended onlyOneHotFromOrdinalTransformer, which by design requires input that is already ordinalencoded. The
Preprocessorhands it raw column values, so any string column died insidenp.max(X, axis=0).astype(int):Nothing in that message points at the chosen method. And since
_detect_column_typesroutesevery non-numeric column to the categorical side, this was the common case, not an edge one.
Three docs disagreed about what the method does:
preprocessor.py:56docs/user_guide/preprocessing.md:80README.md:107Fix
Insert
ContinuousOrdinalTransformerahead ofOneHotFromOrdinalTransformerin thecategorical pipeline, making the behaviour match the
Preprocessordocstring — which isalso the reading that makes the method usable from the
Preprocessorat all.The alternative (guard and raise, pointing users at
"one-hot") would leave the methodunreachable through its own entry point, so I went with encoding.
Behaviour worth reviewing
ContinuousOrdinalTransformernumbers categories from1and reserves0for unseenvalues. Two consequences, both of which I think are the right outcome but which are
user-visible:
kcategories expands tok + 1columns;0rather than producing anall-zero row (which is what plain
handle_unknown="ignore"one-hot does).Both are now documented in the
Preprocessordocstring and the user-guide table.Tests
Four added to
tests/test_categorical_pipeline.py: the reported string case, pipeline stepordering, unseen categories landing in the reserved column, and
get_feature_names_out()width agreeing with
transform.Full suite: 484 passed, 9 xfailed.
ruff checkclean on changed files; pyright unchanged atits pre-existing 71 errors.
🤖 Generated with Claude Code