[high] Exit non-zero and keep converting when a rule fails with --output-dir

[elhoim]

BLUF

• Problem: with --output-dir, any conversion error is downgraded to a warning and the command exits 0. Every rule after the failing one is silently dropped.
• Impact: sigma convert -t <backend> -od out/ rules/ in a CI job reports success while out/ is incomplete, and can even be empty. The same input without -od exits 1.
• Fix: convert rule by rule in write_separate_files(), report each failing rule, keep writing the others, and exit 1 when any rule failed.
• Tests: two new CliRunner tests. The first (fail rule + good rule → exit 1, good rule still written) fails on main and passes with the fix. The second guards --skip-unsupported (still exit 0, reported under "Ignored errors").

Priority: high

Details

Root cause: sigma/cli/convert.py:192-195

try:
    backend.convert(rule_collection, format, correlation_method, callback=write_callback)
except Exception as e:
    click.echo(f"Warning: Failed to convert rules: {e}", err=True)

Backend.convert() is a single list comprehension over all rules. The first exception aborts it, so the callback never runs for later rules. The exception is then swallowed. The single-output path (convert.py:590-601) turns the same SigmaError / NotImplementedError into Error while converting: ... with exit status 1.

The fix does what Backend.convert() does: init_processing_pipeline() and resolve_rule_references(), then convert_rule() / convert_correlation_rule() in collection order with the existing callback. The difference is that SigmaError / NotImplementedError is caught per rule. finalize() is not called, but the old code discarded its return value anyway. After the files are written, a ClickException reports the number of failed rules.

-s/--skip-unsupported still works as before. In that mode the backend collects SigmaErrors itself (collect_errors=True), and they are listed under "Ignored errors" with exit 0.

Before (rule a.yml uses an unresolvable |expand placeholder, b.yml is fine):

$ sigma convert -t text_query_test -od out rules/
Warning: Failed to convert rules: Attempt to convert unhandled placeholder 'var' into query.
Wrote 0 file(s) to out
$ echo $?
0

After:

Error: Failed to convert rule /tmp/demo/rules/a.yml: Attempt to convert unhandled placeholder 'var' into query.
Wrote 1 file(s) to out
Error: 1 rule(s) failed to convert, see errors above.
$ echo $?
1

This PR is independent of the other --output-dir PR, which changes the write loop of the same function. Whichever lands second, I'll rebase it.

Testing

• New tests: test_convert_output_dir_conversion_error_fails_and_continues (fails on main, passes here) and test_convert_output_dir_conversion_error_skip_unsupported.
• Full suite, run the way CI runs it (Python 3.12, dependencies pinned to poetry.lock: pySigma 1.4.0, click 8.4.2, pyparsing 3.3.2): pytest --cov=sigma → 120 passed, 1 skipped.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟡 Changes recommended

The new tests assert stderr-emitted messages via result.output, which can be incorrect under Click’s separated stdout/stderr capture, risking flaky or failing tests.

Review effort: Lite
Findings: 1 · 1

Open (2)

• Assert stderr for Click error output · New
• Remove redundant resolve_rule_references call · New

What changed in this PR

This PR fixes sigma convert --output-dir so conversion errors are no longer silently downgraded to warnings with exit code 0, and so later rules still get converted/written even if an earlier rule fails—making the command safer for CI usage.

Changes:

• Convert rules one-by-one in write_separate_files() to continue after per-rule conversion failures.
• Exit non-zero when any rule fails to convert (while still writing outputs for successful rules).
• Add CLI tests covering “fail + continue” behavior and --skip-unsupported behavior with --output-dir.

File	Description
sigma/​cli/​convert.py	Switches --output-dir conversion to per-rule processing and raises an error if any rule fails.
tests/​test_convert.py	Adds regression tests for non-zero exit on conversion failure while still writing later rules, plus --skip-unsupported guard.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.