Skip to content

fix(parse): count dropped facts, and keep the ones whose undeclared context the filing dates itself - #113

Merged
jfrench9 merged 1 commit into
mainfrom
bugfix/undeclared-context-facts
Nov 19, 2026
Merged

jfrench9 merged 1 commit into
mainfrom
bugfix/undeclared-context-facts

Conversation

@jfrench9

Copy link
Copy Markdown
Member

Summary

Refs #111 (steps 1 and 2 of the plan there; umbrella #60).

A fact whose contextRef names an id the instance never declares was skipped in to_xbrl_model with no warning and no count. Some NSE quarterly-results filings put nearly every fact under two such ids (OneD, FourD), so the model held 4 facts of 99.

Never silent. Every fact the walk leaves out is counted on a new model.parse_report, by reason (undeclared_context, undefined_concept, invalid_period) and context id, with a sample of the concepts. It is logged as one warning and shown in the load_filing receipt and describe_filing. parse_report is None when there is nothing to say, so an ordinary filing's output is unchanged.

Recovered where the filing allows it. An undeclared context is rebuilt when the filing reports exactly one DateOfStartOfReportingPeriod and one DateOfEndOfReportingPeriod under that same id. The rules are narrow:

  • The period comes only from those two facts, never from a declared sibling context (the affected filings date some of those wrongly).
  • The entity is the one every declared context names. If they disagree nothing is rebuilt; if the instance declares none, the caller's filer id stands in.
  • Only duration concepts are placed, with no dimensions.

A recovered fact carries provenance.kind == "recovered_context". The holon and TAVI projections already write FactProvenance and their readers restore it, so the mark survives an export with no serializer change.

Results

Filing Before After
A, KSCL Dec 2022 quarter 4 of 99 99 of 99 (52 under OneD, 43 under FourD)
B, ABAN Sep 2020 quarter 4 of 188 188 of 188 (143 and 41)
C, control 306 of 306 306 of 306, no parse report

These three were run against a stub schema generated from each instance, since the NSE taxonomy is not bundled yet. Types and labels are therefore not the real ones; the fact counts, periods and entity are. A run against the real taxonomy is still wanted.

Things a reviewer should know

  • Fact ids for recovered facts are minted differently. Arelle hashes a fact that has no context by its qname alone, so every orphan of one concept shares a hash (filing B collided on two cash-flow facts). Recovered facts fold the context id, value, language, nil and unit into the hash. Declared facts are untouched.
  • New model surface: XbrlModel.parse_report and the ParseReport, DroppedFacts, RecoveredContext types. Additive; parse_report does not travel through holon or TAVI, only the per-fact mark does.
  • Not in this PR: the declared Four… contexts that carry a quarter's dates on a year-to-date value are left exactly as the filing declares them, and taxonomy packaging is separate. Both are on Indian filings (NSE / BSE results, Ind AS) load cleanly in xbrlkit #60's checklist.
  • The rule is keyed on two concept local names (_REPORTING_PERIOD_FACTS). It is a filing pattern, not a standard.

Testing

tests/test_parse_contexts.py: 12 tests on synthetic instances that load offline through Arelle, covering recovery, each refusal (half a period, conflicting dates, instant concept, disagreeing entities), distinct ids, the round trip through holon and TAVI, and the receipt. Full suite: 735 collected, 733 passed, 2 skipped; ruff, format and basedpyright clean.

🤖 Generated with Claude Code

…n the filing states the period

A fact whose contextRef names an id the instance never declares was skipped
with no warning and no count. Some Indian exchange results filings put nearly
every fact under two such ids, so the model held 4 facts of 99.

Every fact the walk leaves out is now counted on model.parse_report by reason
and context id, logged, and shown in the load receipt and describe_filing.

An undeclared context is rebuilt when the filing reports exactly one start
and one end date under that id; its facts carry provenance kind
"recovered_context", which the holon and TAVI projections already keep.

Refs #111
@jfrench9
jfrench9 merged commit 9fff733 into main Nov 19, 2026
6 checks passed
@jfrench9
jfrench9 deleted the bugfix/undeclared-context-facts branch November 19, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant