[high] Report collection errors and broken filters in sigma check

[elhoim]

BLUF

• Problem: sigma check only reads the errors attached to rules in SigmaCollection.rules. It ignores errors that belong to no rule, such as an unknown action: or an invalid filter (filters are stored in SigmaCollection.filters).
• Impact: sigma check --fail-on-error --fail-on-issues rules/ passes (exit 0, "Found 0 errors") on a file that sigma convert -t <backend> rules/ rejects with "Errors found in Sigma rules". A CI gate built on sigma check lets these files through.
• Fix: load_and_check_rules() now also reports the entries of SigmaCollection.errors that don't belong to a loaded rule. They count as rule errors, so --fail-on-error applies, and they are added to the --junitxml report. Per-rule errors, which pySigma also copies into SigmaCollection.errors, are still counted once.
• Tests: four new CliRunner tests: unknown action, invalid filter, JUnit output, and no double counting. The first three fail on main and pass with the fix.

Priority: high

Details

Root cause: sigma/cli/check.py:130-175. load_and_check_rules() loops over rule_collection.rules and reads rule.errors. It never reads rule_collection.errors, but sigma convert does, via check_rule_errors() (sigma/cli/rules.py:39-49). The two commands therefore disagree on:

• Collection-level errors. For example action: repaet raises SigmaCollectionError. No rule is created, so the error lives only in SigmaCollection.errors.
• Filter errors. A SigmaFilter is kept in SigmaCollection.filters, not in .rules, so its errors (for example an invalid id) are never seen.

pySigma also puts every per-rule error into SigmaCollection.errors (collection.py errors.extend(parsed_rule.errors)). The fix therefore skips errors that are already attached to a rule in .rules, compared by object identity. The count for tests/files/invalid stays at 6.

Before:

$ printf 'action: repaet\ntitle: typo in action keyword\n' > rules/typo.yml
$ sigma check --fail-on-error --fail-on-issues rules/
Found 0 errors, 0 condition errors and 0 issues.
$ echo $?
0

After:

=== Sigma Rule Errors ===
Unknown Sigma collection action 'repaet' in rule 1 in rules/typo.yml

=== Summary ===
Found 1 errors, 0 condition errors and 0 issues.
...
Check failure
$ echo $?
1

Testing

• New tests in tests/test_check.py:
  • test_check_unknown_collection_action (fails on main)
  • test_check_invalid_filter (fails on main)
  • test_check_collection_error_junitxml (fails on main)
  • test_check_rule_errors_not_double_counted (regression guard)
• Full suite, run the way CI runs it (Python 3.12, dependencies pinned to poetry.lock: pySigma 1.4.0, click 8.4.2, pyparsing 3.3.2): pytest --cov=sigma → 122 passed, 1 skipped.

🤖 Generated with Claude Code