fix(preprocessor): raise a clear error on duplicate column names - #43
Open
ChrisW09 wants to merge 1 commit into
Open
fix(preprocessor): raise a clear error on duplicate column names#43ChrisW09 wants to merge 1 commit into
ChrisW09 wants to merge 1 commit into
Conversation
Fitting on a DataFrame with a repeated column label failed with
AttributeError: 'DataFrame' object has no attribute 'dtype'
from inside ``_detect_column_types``, with nothing pointing at the cause:
``X[col]`` returns a DataFrame rather than a Series for a duplicated label, so
the dtype inspection has nothing to read. (``nunique()`` on the line above
happens to work on a frame, which is why it surfaced one line later.)
Check for duplicates up front and raise ``PretabDataError`` naming them. The
ColumnTransformer this builds keys its transformers by column name, so
duplicates could not be routed unambiguously even if detection coped with them
-- rejecting them early is the honest outcome.
Closes #37
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChrisW09
force-pushed
the
fix/duplicate-column-names
branch
from
July 27, 2026 20:50
a3c4e02 to
49863eb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #37.
Problem
X[col]returns a DataFrame rather than a Series whencolis duplicated, so the dtypeinspection has nothing to read. (
nunique()on the preceding line happens to work on aframe, which is why the failure surfaces one line later and looks unrelated.)
Fix
Detect duplicates up front and raise a typed error naming them:
Rejecting rather than accommodating is the honest outcome here: the
ColumnTransformerthisbuilds keys its transformers by column name (
f"num_{feature}"), so duplicate labels couldnot be routed unambiguously even if type detection coped with them.
Tests
Three added to
tests/test_exceptions.py: the reported single-duplicate case, a frame withtwo distinct duplicated labels (asserting both are listed), and a unique-column frame to
confirm the check is not over-eager.
Full suite: 483 passed, 9 xfailed.
ruff checkclean on changed files; pyright unchanged atits pre-existing 71 errors.
🤖 Generated with Claude Code