Skip to content

feat(dns): forward DNS queries and error responses to rule evaluation - #1028

Open
slashben wants to merge 2 commits into
mainfrom
SUB-9110-dns-query-rule-forwarding
Open

slashben wants to merge 2 commits into
mainfrom
SUB-9110-dns-query-rule-forwarding

Conversation

@slashben

@slashben slashben commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The DNS tracer parses every DNS packet but forwarded only responses with at least one answer, so a query that gets NXDOMAIN or no reply at all (the shape of one-way DNS exfiltration) never reached a rule. This adds an opt-in flag, dnsRuleForwarding, that forwards queries and error responses to the rule manager only, exposes qtype/qr/numAnswers/rcode to CEL, and keeps existing dns rules untouched via a dnsQuery opt-in expression type. Flag off is behaviour-identical to today.

Ticket

SUB-9110

Changes

  • pkg/containerwatcher/v2/tracers/dns.go: WithDNSRuleForwarding option, pure classifyDNS (answered → as before; query / NXDOMAIN / SERVFAIL / REFUSED → forwarded rule-only; NODATA, empty names and .cluster.local. / .in-addr.arpa. / .ip6.arpa. / configured suffixes → dropped at the tracer, before enqueue and enrichment). The subscribe callback now classifies in place and DeepCopys only forwarded packets, removing a copy per dropped packet. Counter node_agent.dns.rule_forwarding.total{outcome}.
  • pkg/utils: RuleOnlyEvent marker on DatasourceEvent and StructEvent; RuleEventType() returns the alias dnsQuery for rule-only DNS events; DNSEvent.GetQtype/GetRcode; CEL fields qtype, qr, numAnswers, rcode (has(event.rcode) false on a query).
  • pkg/containerwatcher/v2/event_handler_factory.go: ruleOnlyHandlers list (dns → rule manager only), rule-only events skip third-party receivers and are dropped when duplicate; DNS dedup case for rule-only events only (5s, key includes pid, qr, qtype). Answered responses are never deduplicated, as before.
  • pkg/rulemanager: rule expressions are selected by utils.RuleEventType in ReportEnrichedEvent and CEL.EvaluateRule; eval-context eventType stays dns.
  • pkg/rulemanager/ruleadapters/adapters/dns.go: rule-only alerts carry qr, qtype, numAnswers, rcode, nameserver arguments; answered-response alerts unchanged.
  • pkg/config: DNSRuleForwardingConfig{Enabled, ExcludedSuffixes}, default off, suffixes normalized on load; only wired when runtimeDetectionEnabled.
  • Docs: docs/features/dns-query-rule-forwarding.md, docs/CONFIGURATION.md.

Why the dnsQuery alias

About nine rules are written for eventType: "dns" on the assumption the event is an answered response. With ndots:5, api.github.com.ec2.internal. and friends are queried on every external lookup and never enter the learned egress baseline, so routing queries to dns expressions would fire R0005 in every pod. A guard such as event.qr == 'R' cannot be retrofitted either: an older agent disables an expression that references a field it does not know. Rules opt in by adding a dnsQuery expression; older agents load it and never match it.

Not in this PR

  • Helm chart key (nodeAgent.config.dnsRuleForwarding); until then use the nodeAgent.config.extra passthrough.
  • The host watcher in the private agent constructs the same tracer and stays on the historical filter until it passes the option itself, after its dispatch loop learns rule-only routing.

Testing

go test ./pkg/rulemanager/... ./pkg/containerwatcher/v2/ ./pkg/utils/... ./pkg/dedupcache/ ./pkg/config/ → 1030 passed. go test ./pkg/containerwatcher/v2/tracers/ -skip 'Fields$' → 96 passed (the *Fields gadget tests need tracers.tar, absent locally, and fail identically on main). New tests: TestClassifyDNS (flag on/off × packet kinds × suffixes, plus a real datasource packet), TestEventHandlerFactory_DNS* (rule-only reaches the rule manager only; answered reaches all four handlers and third-party; dedup marks only rule-only events), TestCEL_DNS* (fields, has(event.rcode), alias routing both directions), TestDnsAdapter_*, TestLoadConfig_DNSRuleForwarding*, updated ComputeDNSKey tests. go vet clean.

Docs

docs/features/dns-query-rule-forwarding.md (new), docs/CONFIGURATION.md (DNS Rule Forwarding section).

AI Context

Category Used
Skills agent-dispatch-policy, writing-docs
MCP Servers None
Rules/Commands docs-gate, pr-structure, workspace CLAUDE.md

AI-skills: armosec-shared-rules:agent-dispatch-policy | cmds: /model

Summary by CodeRabbit

  • New Features
    • Added optional DNS rule forwarding, disabled by default and requiring runtime detection. DNS queries and error responses can now be evaluated by rules using the dnsQuery event type.
    • Added configurable DNS suffix exclusions and DNS query details in rule alerts. Existing handling for answered DNS responses remains unchanged.
  • Documentation
    • Added setup guidance, configuration examples, supported-event details, operational metrics, and forwarding limitations.

The DNS tracer parsed every packet but forwarded only responses with at
least one answer, so a query that gets NXDOMAIN or no reply at all (the
shape of one-way DNS exfiltration) never reached a rule.

Behind the new dnsRuleForwarding flag (default off, behaviour unchanged
when off) the tracer also forwards queries and error responses
(NXDOMAIN, SERVFAIL, REFUSED), marked rule-only:

- rule-only events reach the rule manager only, not the DNS manager
  (which would issue its own lookup), the network stream, metrics,
  profile learning or third-party receivers
- they match rule expressions declared as eventType "dnsQuery", so the
  existing "dns" rules (R0005 in particular) keep seeing answered
  responses only and do not fire on ndots search-suffix churn; older
  agents load "dnsQuery" expressions harmlessly
- NODATA responses and names under .cluster.local., .in-addr.arpa.,
  .ip6.arpa. and configured suffixes are dropped at the tracer, before
  enqueue and enrichment
- rule-only events are deduplicated for 5s on (mntns, pid, name, qr,
  qtype); answered responses are still never deduplicated
- new CEL fields on DNS events: qtype, qr, numAnswers, rcode (responses
  only, has() is false on a query)
- alerts on rule-only events carry qr, qtype, numAnswers, rcode and
  nameserver in their arguments
- the tracer now classifies the packet in place and copies only what it
  forwards, removing a DeepCopy per dropped packet
- counter node_agent.dns.rule_forwarding.total{outcome=...}

Docs: docs/features/dns-query-rule-forwarding.md, docs/CONFIGURATION.md.
Signed-off-by: Ben <ben@armosec.io>
@slashben slashben added the ai-assisted Created through Armosec AI tooling (armosec-shared-rules plugin) label Oct 12, 2026
@coderabbitai

coderabbitai Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d24fcebb-ab16-431d-8694-2f65ae083501
📥 Commits

Reviewing files that changed from the base of the PR and between 2eafeaa and 4064b5b.

📒 Files selected for processing (7)
  • docs/features/dns-query-rule-forwarding.md
  • pkg/containerwatcher/v2/event_handler_factory.go
  • pkg/containerwatcher/v2/event_handler_factory_dns_test.go
  • pkg/dedupcache/keys.go
  • pkg/dedupcache/keys_test.go
  • pkg/rulemanager/cel/cel_dns_test.go
  • pkg/utils/datasource_event.go

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

Adds optional DNS rule forwarding. When enabled, eligible queries and DNS error responses reach rule evaluation as rule-only events. Answered responses retain their existing path. The change also adds DNS rule fields, event deduplication, configuration, and documentation.

Changes: DNS Rule Forwarding

Layer / File(s) Summary
Configuration and event contracts
pkg/config/config.go, pkg/config/config_test.go, pkg/utils/events.go, pkg/utils/datasource_event.go, pkg/utils/struct_event.go, docs/CONFIGURATION.md
Adds disabled-by-default forwarding configuration and normalizes excluded suffixes. DNS event accessors expose query type, response code, and rule-only status. The dnsQuery alias selects rule expressions for rule-only DNS events.
DNS packet classification
pkg/containerwatcher/v2/tracers/dns.go, pkg/containerwatcher/v2/tracers/tracer_factory.go, pkg/containerwatcher/v2/tracers/dns_classify_test.go, docs/features/dns-query-rule-forwarding.md
The tracer classifies packets before copying them. It forwards eligible queries and error responses as rule-only events, while skipping empty names, excluded suffixes, and successful responses without answers. Answered responses remain regular events.
Rule-only dispatch and DNS deduplication
pkg/containerwatcher/v2/event_handler_factory.go, pkg/containerwatcher/v2/event_handler_factory_dns_test.go, pkg/dedupcache/keys.go, pkg/dedupcache/keys_test.go, docs/features/dns-query-rule-forwarding.md
Rule-only DNS events go to the rule manager and bypass third-party receivers. The five-second deduplication key includes mount namespace, PID, DNS name, query/response direction, query type, and response code.
DNS rule evaluation and alert fields
pkg/rulemanager/cel/*, pkg/rulemanager/rule_manager.go, pkg/rulemanager/ruleadapters/adapters/dns.go, pkg/rulemanager/ruleadapters/adapters/dns_test.go, pkg/utils/cel.go, docs/features/dns-query-rule-forwarding.md
Rule evaluation selects expressions using the rule event type while retaining the tracer event type in its context. CEL exposes DNS fields, and rule-only DNS alerts include packet and nameserver fields.

Priority: ⬇️ Low

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant DNSPacket
  participant DNSTracer
  participant EventHandlerFactory
  participant RuleManager
  participant CEL
  DNSPacket->>DNSTracer: Provide DNS packet
  DNSTracer->>EventHandlerFactory: Send eligible rule-only event
  EventHandlerFactory->>RuleManager: Dispatch nonduplicate event
  RuleManager->>CEL: Select expression using dnsQuery rule type
Loading

Suggested reviewers: matthyx

Merge Risk: ⚪ Minimal · up to 4064b

No actionable merge-blocking issue was identified; the change is ready for normal merge checks.

Pre-merge checks · Passed 4 · Failed 1

Failed Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 36.96% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 46 functions across 18 files. (1 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks (4 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the primary change: forwarding DNS queries and error responses to rule evaluation.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 36.96% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 46 functions across 18 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

Performance Benchmark Results

Node-Agent Resource Usage
Metric BEFORE AFTER Delta
Avg CPU (cores) 0.157 0.158 +0.4%
Peak CPU (cores) 0.167 0.164 -1.8%
Peak CPU p95 (cores) 0.164 0.164 -0.2%
Avg Memory (MiB) 409.754 329.266 -19.6%
Peak Memory (MiB) 415.555 342.961 -17.5%
Dedup Effectiveness

No data available.

@matthyx matthyx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes for one confirmed detection false negative.

Reviewed head 2eafeaac96b68faef5aa13d1e5c68372ee4119e6 against main at c7a73937cb2af722c5fce3c9e6599145ea071036 (20 files). The feature is needed: the target tracer drops every query and zero-answer response. Opt-in forwarding addresses that gap, preserves answered-response consumers, and keeps existing dns rules isolated through dnsQuery. Independent correctness and architecture reviews found routing, adapter selection, copied-packet ownership, and release paths sound.

Blocker (P2): the DNS dedup key omits rcode, so a distinct matching error response can disappear before evaluation; see inline finding. An isolated regression reproduces SERVFAIL followed by NXDOMAIN with the same pid/name/type: expected one NameError event at the rule manager, actual zero. Adding rcode to the key in a disposable validation copy makes the regression pass while identical-response dedup still passes. No source-branch changes were made.

Nonblocking: new non-DNS CEL qr returns Q on DatasourceEvent, contrary to the documented empty default; the test covers StructEvent only. Guard non-DNS accessors and add a datasource test when convenient.

History: #247 intentionally filtered unresolved DNS for existing consumers; rule-only dispatch preserves that boundary. #781 removed unhelpful DNS dedup; answered responses remain exempt here. Closed #356 changed old rules to query-only; it had an approval and no stated closure rationale, so closure is not evidence of rejection. #846 was explicitly closed because #806 merged; those address DNS wildcard matching, not forwarding. Open #875 supplies correlation state, a complementary capability. No duplicate forwarding change found.

Searches covered open/closed/merged PRs and issues using DNS, dns.go, NXDOMAIN, exfiltration, dnsQuery, ComputeDNSKey, and forwarding/rule-only variants, capped at 100 results per query; broad rule-only search reached the cap. GitHub text search does not exhaust diffs or private ticket history.

Validation: credential-free, network-isolated bubblewrap execution: full tests for config, dedupcache, utils/..., rulemanager/..., containerwatcher/v2 and tracers passed, excluding gadget *Fields tests (missing tracers.tar). Targeted new DNS tests, go vet for those packages, gofmt and diff --check passed. All 39 current GitHub checks succeeded. No live kernel/eBPF or enabled-feature load test was run.

// parallel A/AAAA retries repeat them within seconds.
case utils.DnsEventType:
if e, ok := event.(utils.DNSEvent); ok && utils.IsRuleOnlyEvent(event) {
return dedupcache.ComputeDNSKey(mntns, e.GetPID(), e.GetDNSName(), string(e.GetQr()), e.GetQtype()), dedupTTLDNS, true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve distinct DNS response codes in the dedup key. With forwarding and default dedup enabled, a process receiving SERVFAIL and then NXDOMAIN for the same name/qtype within five seconds gets identical keys. The second response is discarded before rule evaluation, so has(event.rcode) && event.rcode == 'NameError' never sees its matching event. This can occur when retrying another configured resolver. I reproduced this using the real dispatch fixture and DedupCache: zero NameError events reached the rule manager; including rcode in a disposable key experiment restored delivery. Add rcode to ComputeDNSKey and cover SERVFAIL → NXDOMAIN, alongside repeated-identical-response dedup.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 4064b5b: rcode is now part of ComputeDNSKey (empty on queries, so the query key is unchanged). Added TestEventHandlerFactory_DNSDedupKeepsDistinctResponseCodes on the dispatch fixture (SERVFAIL then NXDOMAIN both delivered, a repeated NXDOMAIN absorbed), TestComputeDNSKey_ResponseCodesAreDistinct, and the rcode variant in the key test. The nonblocking qr point is in the same commit: DatasourceEvent.GetQr now returns empty for non-DNS events like GetQtype and GetRcode, with a datasource-backed exec case in TestCEL_DNSFields_NonDNSEventIsEmpty. Feature doc updated.

…on-DNS events

A SERVFAIL from the first configured resolver and the NXDOMAIN from the
retry share name, type and qr, so the NXDOMAIN was marked Duplicate and a
rule gated on event.rcode never saw it. rcode is now part of
ComputeDNSKey; identical responses still collapse.

DatasourceEvent.GetQr returned Q for every non-DNS event, contrary to the
documented empty value; it now checks the event type like GetQtype and
GetRcode, and the CEL test covers a datasource-backed exec event.

Signed-off-by: Ben Hirschberg <ben@armosec.io>
Signed-off-by: Ben <ben@armosec.io>
@slashben
slashben requested a review from matthyx October 13, 2026 12:05
@github-actions

Copy link
Copy Markdown

Performance Benchmark Results

Node-Agent Resource Usage
Metric BEFORE AFTER Delta
Avg CPU (cores) 0.246 0.245 -0.3%
Peak CPU (cores) 0.259 0.253 -2.3%
Peak CPU p95 (cores) 0.256 0.252 -1.8%
Avg Memory (MiB) 380.665 314.853 -17.3%
Peak Memory (MiB) 388.086 322.199 -17.0%
Dedup Effectiveness

No data available.

@matthyx matthyx left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after re-reviewing 4064b5bdeedfa8f5ed62d31e0307ae20aa84f14a against main at c7a73937cb2af722c5fce3c9e6599145ea071036.

The previous P2 blocker is resolved: DNS dedup now includes rcode, so SERVFAIL followed by NXDOMAIN reaches rule evaluation as two distinct responses while repeated identical responses remain deduplicated. My original failing reproduction now passes on this head. The author added both dispatch-level and key-level regression tests. The nonblocking non-DNS qr default is also corrected and covered with a datasource-backed CEL test.

Reviewed all seven follow-up files with independent correctness and architecture checks. No remaining material blockers found. The opt-in forwarding feature remains needed, existing answered-DNS consumers/rules remain isolated, and routing and packet ownership remain sound. Related history and bounded search limitations are documented in the initial review; refreshed dnsRuleForwarding/NXDOMAIN searches found no superseding change.

Fresh credential-free, network-isolated validation passed: go test ./pkg/config ./pkg/dedupcache ./pkg/utils/... ./pkg/rulemanager/... ./pkg/containerwatcher/v2 ./pkg/containerwatcher/v2/tracers -skip 'Fields$' -count=1, the original reproduction and new targeted regressions, go vet for those packages, gofmt, and diff --check. All 39 current GitHub checks succeeded. Gadget *Fields tests remain excluded because tracers.tar is absent; live eBPF and enabled-feature load testing were not performed. No source branch changes were made.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Created through Armosec AI tooling (armosec-shared-rules plugin)

Projects

Status: Waiting on Author

Development

Successfully merging this pull request may close these issues.

2 participants