Skip to content

Preserve finalizer configurations when loading processing pipelines - #597

Open
leela-padala wants to merge 1 commit into
SigmaHQ:mainfrom
leela-padala:fix/preserve-finalizer-configurations
Open

leela-padala wants to merge 1 commit into
SigmaHQ:mainfrom
leela-padala:fix/preserve-finalizer-configurations

Conversation

@leela-padala

Copy link
Copy Markdown

Loading a processing pipeline with finalizers mutates the caller's configuration by removing type and policy-related keys. Reusing that dictionary for a second pipeline then fails, which affects callers building multiple backend pipelines from a shared parsed configuration.

Minimal reproduction on current main (54e4077):

from sigma.processing.pipeline import ProcessingPipeline

config = {"finalizers": [{"type": "concat", "separator": "|"}]}
ProcessingPipeline.from_dict(config).finalize(["first", "second"])
ProcessingPipeline.from_dict(config)  # SigmaConfigurationError: Finalizer type must be specified

Copy each finalizer entry before parsing it, both in ProcessingPipeline.from_dict and NestedFinalizer.from_dict. The existing policy-key stripping and policy enforcement still apply to the local copy. No public API or output format changes.

Regression tests cover repeated loading and unchanged input for concat, inline templates, nested/recursive finalizers, direct nested-finalizer construction, and rejected template-policy overrides. The original top-level test and four nested cases failed before their respective fixes.

Validation:

  • Focused pipeline/finalizer tests: 134 passed (126 passed on the unchanged baseline).
  • Offline suite: 1,827 passed, one skipped, 11 network tests deselected (pytest -q -m 'not online').
  • black --check .: passed; mypy: no issues in 70 source files; git diff --check: passed.

AI assistance: Codex helped investigate, implement, and test this fix.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant