fix(alertmanager): send resolved notifications to the AI-SRE relay [JDWLABS-476] - #420
Open
jdwlabs-agent-bot[bot] wants to merge 1 commit into
Open
fix(alertmanager): send resolved notifications to the AI-SRE relay [JDWLABS-476]#420jdwlabs-agent-bot[bot] wants to merge 1 commit into
jdwlabs-agent-bot[bot] wants to merge 1 commit into
Conversation
The relay opens one Jira ticket per alert fingerprint and skips investigating repeats while that ticket is open. Without resolved notifications it never learns the condition cleared, so tickets stayed open until a person closed them and every later firing of the same alert was absorbed as a repeat. The relay now closes its own ticket after an alert stays resolved for a grace period, which only works if Alertmanager delivers the resolve; the discord receiver already had this on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FvYBM6o7wARm2v9jmDp1yY
jdwlabs-agent-bot
Bot
requested review from
jdwillmsen and
jdwlabs-root
as code owners
September 5, 2026 03:23
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Flips
send_resolvedtotrueon theai-sreAlertmanager receiver so the relay receives resolved notifications, and records why in the adjacent comment. One line of config; thediscordreceiver in the same file already has it on.Why
The relay opens one Jira ticket per alert fingerprint and skips investigating repeats while that ticket is open. It never learned that a condition had cleared, so tickets stayed open until a person closed them, and every later firing of the same alert name was absorbed as a repeat. This week that left eleven stale tickets open, made the relay investigate nothing for 12h (
AiSreRelayInvestigatingNothing), and produced an auto-remediation PR proposing to disable dedup entirely.The relay-side feature that consumes this notification is jdwlabs/apps#213: note the resolve time on the ticket, close it after the alert stays resolved for a configurable grace (default 6h), cancel on re-fire, and never let a Done ticket suppress a new investigation. Merge order does not matter: the relay currently drops resolved alerts at the webhook, so enabling this first is inert until #213 deploys.
Verification
python3 -m unittest discover -s tools/tests -t tools/testspasses locally.tests/alertmanager-routing) is unaffected: receivers and routes are unchanged, only a delivery flag on an existing webhook config.Review gate
tenants/platform/is a CODEOWNERS-gated path, so this PR is authored under thejdwlabs-agent-botidentity to allow a human codeowner approval without the self-review wall.🤖 Generated with Claude Code
https://claude.ai/code/session_01FvYBM6o7wARm2v9jmDp1yY