Skip to content

fix(transformers): handle DataFrame input in ContinuousOrdinalTransformer - #26

Open
ChrisW09 wants to merge 1 commit into
mainfrom
fix/continuous-ordinal-2d-input
Open

fix(transformers): handle DataFrame input in ContinuousOrdinalTransformer#26
ChrisW09 wants to merge 1 commit into
mainfrom
fix/continuous-ordinal-2d-input

Conversation

@ChrisW09

Copy link
Copy Markdown
Collaborator

Fixes #14.

Problem

fit iterated X.T and transform iterated X. Neither does what it looks like for a
DataFrame: DataFrame.T iterates the column labels of the transpose (the original row
index), and iterating a DataFrame yields column names. So a frame produced one mapping per
row and an all-zero result of the wrong shape — with no exception:

df = pd.DataFrame({"c1": ["a", "b", "a", "c"], "c2": ["x", "y", "x", "y"]})
t = ContinuousOrdinalTransformer().fit(df)
len(t.mapping_)   # 4, should be 2 — keyed by the row index
t.transform(df)   # [[0 0], [0 0]] — should be (4, 2) with real codes

The Preprocessor path was never affected, because the SimpleImputer ahead of this step
always hands it an ndarray. But the transformer is exported from pretab.transformers and
documented in the API reference, and direct frame users got plausible-looking garbage.

Fix

Normalize to a 2D object array in both methods (_as_2d) and index by position.

Two related fixes carried in the same change, both in the issue:

  • transform now allocates np.zeros(X.shape, dtype=int) instead of building a list
    comprehension, so zero rows return (0, n_features) rather than collapsing to a 1-D
    (0,) array that breaks any downstream hstack.
  • transform validates the feature count and raises PretabDataError, matching every other
    transformer in the package (which get this from validate_2d_allow_nan).

Result

mapping_ length : 4  ->  2
transform(df)   : [[0 0], [0 0]]  ->  [[1 1], [2 2], [1 1], [3 2]]   shape (4, 2)
empty transform : (0,)  ->  (0, 2)

DataFrame, ndarray and plain-list input now all agree.

Tests

New file tests/test_continuous_ordinal_transformer.py (matching the one-file-per-transformer
convention already used by test_custombin_transformer.py and friends), 7 tests: per-column
mappings, frame codes, frame/ndarray equivalence, list input, the empty-input shape, the
feature-count guard, and a check that unknown categories still map to 0.

Full suite: 487 passed, 9 xfailed. ruff check clean on changed files.

On pyright: the count goes 71 → 72, but the extra entry is Import "pytest" could not be resolved in the new test file. Pyright is configured against a .venv that does not exist
in this checkout, so every test file already produces one of these (34 of them on main).
No new diagnostic in pretab/ itself.

🤖 Generated with Claude Code

…rmer

``fit`` iterated ``X.T`` and ``transform`` iterated ``X``. For a DataFrame the
first yields the column labels of the transpose -- the original row index --
and the second yields column names, so a frame produced one mapping per row
and an all-zero result of the wrong shape. Nothing raised:

    df = pd.DataFrame({"c1": ["a","b","a","c"], "c2": ["x","y","x","y"]})
    ContinuousOrdinalTransformer().fit(df).mapping_   # 4 entries, not 2
    ...transform(df)                                  # [[0 0] [0 0]]

Normalize to a 2D object array in both methods and index by position. The
Preprocessor path was unaffected -- the SimpleImputer ahead of this step always
hands it an ndarray -- but the transformer is public and documented.

Two related fixes while in here:

- ``transform`` builds its output with ``np.zeros(X.shape, dtype=int)`` instead
  of a list comprehension, so zero rows now yield ``(0, n_features)`` rather
  than collapsing to a 1-D ``(0,)`` array that breaks a downstream hstack.
- ``transform`` validates the feature count and raises ``PretabDataError``,
  matching every other transformer in the package.

Closes #14

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(transformers): ContinuousOrdinalTransformer breaks on DataFrames

1 participant