Skip to content

[bugfix] honor sequence_fields on simple sequence features - #675

Merged
tiankongdeguiji merged 4 commits into
alibaba:masterfrom
tiankongdeguiji:chore/grouped-seq-expr-test
Sep 18, 2026
Merged

tiankongdeguiji merged 4 commits into
alibaba:masterfrom
tiankongdeguiji:chore/grouped-seq-expr-test

Conversation

@tiankongdeguiji

@tiankongdeguiji tiankongdeguiji commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

sequence_fields was only emitted for grouped sequence sub-features, so any simple sequence feature (sequence_expr_feature, sequence_lookup_feature, …) whose sequence input is user side silently lost the declaration on its way to the fg json. fg then had no item-side input left to infer the sequence field from and aborted the process with Check failed: !seq_fields_.empty() in sequence_feature.cc — a process abort, not a Python exception.

Fix

Emit sequence_fields whenever the feature is a sequence, in ExprFeature, LookupFeature, MatchFeature, ComboFeature, OverlapFeature and KvDotProduct. The grouped path and the item-side default (e.g. variables: ["user:request_time", "item:event_time"] with no sequence_fields) are unchanged. LookupFeature's second declaration — the one pointing the chained raw bucketizer at the __lookup stub for value_dim > 1 — stays grouped only, since fg resolves that reference on its own for a standalone sequence feature (verified both ways).

Single-input features (id / raw / tokenize / combine / regex_replace) need no change: fg already treats their one non-feature: input as the sequence. CustomFeature also needs none, its operator handles the sequence itself.

Tests

The shape throughout is the classic time-diff request_time - event_time, request_time scalar user-side and event_time the sequence field.

  • expr_feature_test.py: test_sequence_expr_feature_dense / ..._with_boundaries — grouped sub-feature. Because ExprFeature is not in SINGLE_INPUT_FEATURE_CLASSES and both variables are user:-side, the default rule would mark neither as a sequence input; only sequence_fields=["event_time"] makes it resolve to the group-prefixed click_50_seq__event_time while request_time stays scalar. Dense (SequenceDenseData) and bucketized (SequenceSparseData) paths, a string-typed scalar, and null scalar / null sequence rows falling back to default_value.
  • test_simple_sequence_expr_feature_dense and one new simple-sequence test per cross feature (lookup with boundaries, lookup with value_dim=2, match, combo, overlap, kv_dot_product), each parameterized over the item-side default and the user-side field declared through sequence_fields, asserting the emitted fg json and identical parsed output from both shapes.

Docs: a note in the simple-sequence section of docs/source/feature/feature.md on when sequence_fields is needed there.

Test Plan

  • python -m unittest discover -s tzrec/features -p "*_test.py" -t . — 329 tests pass (was 317 before the new cases).
  • python -m unittest tzrec.datasets.data_parser_test tzrec.datasets.dataset_test — 51 tests pass.
  • Each new user-side case aborts the process on the fg CHECK without the fix, rather than failing an assertion.
  • pre-commit run --files ... and pyrefly check clean.

Expected values were taken from live pyfg 1.0.6 runs rather than derived by hand, and the user-side cases are cross-checked against the item-side ones, which produce the same numbers.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Tq3essxgD2zr8RuNxeKUJW

@tiankongdeguiji tiankongdeguiji changed the title [chore] add grouped sequence expr feature test [bugfix] honor sequence_fields on simple sequence expr feature Sep 17, 2026
@tiankongdeguiji tiankongdeguiji changed the title [bugfix] honor sequence_fields on simple sequence expr feature [bugfix] honor sequence_fields on simple sequence features Sep 17, 2026
tiankongdeguiji and others added 4 commits September 17, 2026 20:00
Covers ExprFeature as a grouped sequence sub-feature with a scalar user-side
variable and a sequence user-side variable declared via sequence_fields, for
both the dense and the bucketized output paths.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tq3essxgD2zr8RuNxeKUJW
ExprFeature._fg_json() only emitted sequence_fields for grouped sequence
sub-features, so a simple sequence_expr_feature whose sequence variable is
user side silently lost the declaration. pyfg then had no item-side variable
to infer the sequence field from and aborted the process with a CHECK failure
in sequence_feature.cc. Emit sequence_fields whenever the feature is a
sequence; the grouped path and the item-side default are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tq3essxgD2zr8RuNxeKUJW
Same gating as the expr feature: lookup, match, combo, overlap and
kv_dot_product only emitted sequence_fields for grouped sequence
sub-features, so a simple sequence feature whose sequence input is user
side lost the declaration, leaving fg with no item-side input to infer the
sequence field from, and it aborted the process with a CHECK failure. Emit
sequence_fields whenever the feature is a sequence. The lookup stub chain
declaration stays grouped only, fg resolves that reference on its own for a
standalone sequence feature.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tq3essxgD2zr8RuNxeKUJW
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tq3essxgD2zr8RuNxeKUJW
@tiankongdeguiji
tiankongdeguiji force-pushed the chore/grouped-seq-expr-test branch from 5615b19 to 13f3d39 Compare September 17, 2026 12:01
@tiankongdeguiji tiankongdeguiji added the claude-review Let Claude Review label Sep 17, 2026
@github-actions github-actions Bot removed the claude-review Let Claude Review label Sep 17, 2026
Comment on lines +694 to +697
self.assertEqual(seq_feat.inputs, ["kv_cate", "cate"])
self.assertEqual(
seq_feat.fg_json()[0].get("sequence_fields"), sequence_fields or None
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: this is the only test covering the value_dim>1 stub+raw shape — the one behavior this PR deliberately made asymmetric (sequence_fields on the stub for simple sequences, raw-cfg __lookup declaration grouped-only). Two cheap pins:

  • self.assertEqual(seq_feat.sequence_input_names, ["cate"]) — every sibling simple-sequence test asserts this; the value_dim variant only asserts inputs.
  • self.assertNotIn("sequence_fields", seq_feat.fg_json()[1]) — nothing in the repo asserts anything about fg_json()[1], so a future edit that mis-scopes the raw-cfg declaration to simple sequences too (or drops it in a way the grouped parse test doesn't exercise) would pass CI silently.

The grouped value_dim=2 parse case does catch outright removal of the stub declaration (fg aborts without it), so this is just about pinning the simple-mode contract at the json level.

@github-actions

Copy link
Copy Markdown
Contributor

Review summary

Five review passes ran (code quality, performance, test coverage, docs accuracy, security) — all completed. Overall: the fix is correct, surgical, and well-tested. Details and the few non-blocking suggestions below.

Verified

  • Emission change is sound. is_grouped_sequence → is_sequence is applied identically across the six multi-input classes; since grouped implies sequence, the only added behavior is the intended one (simple sequences with explicit sequence_fields). Grouped and non-sequence fg json is unchanged — for existing working configs the exported json stays byte-identical, so no serving/export compat risk.
  • Lookup restructure is correct. Keeping the raw-cfg __lookup stub declaration grouped-only matches the established tokenize_feature normalizer-stub pattern, and for simple sequences the sequence_* type prefix carries the alignment; the new end-to-end value_dim=2 test exercises both shapes through live pyfg.
  • Excluded classes check out. Single-input classes (id/raw/tokenize/combine/regex_replace) rely on fg single-non-feature:-input inference. CustomFeature/BoolMaskFeature use operator-internal sequence handling: libfg registers no sequence_custom_feature/sequence_bool_mask_feature types, and the pre-existing simple-sequence custom test (both inputs user-side, no sequence_fields) parses correctly through live pyfg. Minor: BoolMaskFeature is the one multi-input class the PR body does not mention — worth a line there for completeness; no code change needed.
  • Docs claims verified against _is_sequence_input/sequence_input_names and the proto; the seq_expr_2 hypothetical is exactly the bug being fixed. Optional precision: the "等" after the six operators could be read as covering simple-sequence custom/bool-mask features, whose fg operators do not consume sequence_fields.
  • Test math independently re-derived (expr time-diff incl. null/boundaries bucketization, match nested-map, kv-dot, overlap ratio, combo) — all consistent; conventions (param + parameterized_name_func) followed throughout.

Suggestions (non-blocking)

  1. Inline on lookup_feature_test.py: pin the stub/raw asymmetry in the value_dim test with a sequence_input_names assertion and assertNotIn("sequence_fields", fg_json()[1]).
  2. Follow-up (not this PR): validate sequence_fields entries against the bare input names of the feature in BaseFeature. A typo (e.g. "user:cate" instead of "cate" — easy, since every doc example writes expressions with the side prefix) now flows verbatim into the C++ fg parser for simple sequences and aborts the process with an opaque Check failed at handler construction. A Python-side subset check would turn that into an actionable error and would cover the grouped path too.

@tiankongdeguiji
tiankongdeguiji merged commit 5c2d616 into alibaba:master Sep 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants