Skip to content

fix(onehot): encode categories before onehot_from_ordinal - #28

Open
ChrisW09 wants to merge 1 commit into
mainfrom
fix/onehot-from-ordinal-encodes-strings
Open

fix(onehot): encode categories before onehot_from_ordinal#28
ChrisW09 wants to merge 1 commit into
mainfrom
fix/onehot-from-ordinal-encodes-strings

Conversation

@ChrisW09

Copy link
Copy Markdown
Collaborator

Fixes #17.

Problem

categorical_method="onehot_from_ordinal" appended only
OneHotFromOrdinalTransformer, which by design requires input that is already ordinal
encoded. The Preprocessor hands it raw column values, so any string column died inside
np.max(X, axis=0).astype(int):

Preprocessor(categorical_method="onehot_from_ordinal").fit(pd.DataFrame({"c": ["a","b","c"]*30}))
# ValueError: invalid literal for int() with base 10: 'c'

Nothing in that message points at the chosen method. And since _detect_column_types routes
every non-numeric column to the categorical side, this was the common case, not an edge one.

Three docs disagreed about what the method does:

source claim
preprocessor.py:56 "integer codes then one-hot"
docs/user_guide/preprocessing.md:80 "One-hot from pre-encoded ordinals"
README.md:107 "One-hot on pre-encoded categoricals"

Fix

Insert ContinuousOrdinalTransformer ahead of OneHotFromOrdinalTransformer in the
categorical pipeline, making the behaviour match the Preprocessor docstring — which is
also the reading that makes the method usable from the Preprocessor at all.

The alternative (guard and raise, pointing users at "one-hot") would leave the method
unreachable through its own entry point, so I went with encoding.

Behaviour worth reviewing

ContinuousOrdinalTransformer numbers categories from 1 and reserves 0 for unseen
values. Two consequences, both of which I think are the right outcome but which are
user-visible:

  • a column with k categories expands to k + 1 columns;
  • a category unseen at fit time lands in the reserved column 0 rather than producing an
    all-zero row (which is what plain handle_unknown="ignore" one-hot does).
3 categories -> shape (90, 4)
['cat_c__c_bin_0', 'cat_c__c_bin_1', 'cat_c__c_bin_2', 'cat_c__c_bin_3']
unseen "ZZZ"  -> [1. 0. 0. 0.]

Both are now documented in the Preprocessor docstring and the user-guide table.

Tests

Four added to tests/test_categorical_pipeline.py: the reported string case, pipeline step
ordering, unseen categories landing in the reserved column, and get_feature_names_out()
width agreeing with transform.

Full suite: 484 passed, 9 xfailed. ruff check clean on changed files; pyright unchanged at
its pre-existing 71 errors.

🤖 Generated with Claude Code

``categorical_method="onehot_from_ordinal"`` appended only
``OneHotFromOrdinalTransformer``, which requires input that is already ordinal
encoded. The Preprocessor hands it raw column values, so any string column died
inside ``np.max(X, axis=0).astype(int)`` with a bare

    ValueError: invalid literal for int() with base 10: 'c'

that says nothing about the chosen method. Since ``_detect_column_types`` routes
every non-numeric column to the categorical side, this was the common case.

Insert ``ContinuousOrdinalTransformer`` ahead of it so the method does what the
Preprocessor docstring already claimed -- integer codes, then one-hot. Because
that encoder numbers categories from 1 and reserves 0 for unseen values, a
column with k categories expands to k + 1 columns and categories unseen at fit
time land in the reserved column instead of producing an all-zero row.

Docs reconciled: the Preprocessor docstring, the user-guide table and the class
itself previously gave three different accounts of this method.

Closes #17

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ChrisW09
ChrisW09 force-pushed the fix/onehot-from-ordinal-encodes-strings branch from ef00389 to 0cb45b0 Compare July 27, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(onehot): categorical_method onehot_from_ordinal fails on strings

1 participant