Skip to content

Verify source_ids field dependencies per PR #20 review feedback - #30

Closed
rjurney with Copilot wants to merge 1 commit into
rjurney/scalefrom
copilot/sub-pr-20-another-one
Closed

rjurney with Copilot wants to merge 1 commit into
rjurney/scalefrom
copilot/sub-pr-20-another-one

Conversation

Copilot AI commented Dec 20, 2025 •

Copy link
Copy Markdown

Addresses review comment r2637286239 from PR #20 requesting verification that no code depends on the source_ids field after its removal from test data.

Analysis Results

Confirmed the codebase correctly treats source_ids as a temporary internal field:

  • BAML layer: source_ids tracks integer ID merges during entity resolution API calls
  • Python ER (abzu/er/uuid.py): Explicitly removes source_ids from output (L402), converts to source_uuids for persistence
  • Spark pipeline: No dependencies on source_ids found
  • Design intent: source_uuids is the persistent provenance field; source_ids exists only within BAML processing scope
# Line 402 in abzu/er/uuid.py
# IMPORTANT: Do NOT include integer IDs (id, source_ids) in output
# These are internal to BAML processing and should not persist between iterations
resolved_dict: dict[str, Any] = {
    "uuid": master_uuid,
    "source_uuids": source_uuids_final,  # Persistent
    # source_ids intentionally omitted
}

Removal of source_ids from final_er.baml test data is correct—that function operates on UUIDs directly without integer ID mapping.

No code changes required. Implementation already handles this correctly.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Update data collection and processing based on feedback Verify source_ids field dependencies per PR #20 review feedback Dec 20, 2025
Copilot AI requested a review from rjurney December 20, 2025 22:08
@rjurney rjurney closed this Dec 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants