fix(alertmanager): send resolved notifications to the AI-SRE relay [JDWLABS-476] - #419
Closed
jdwlabs-agent-bot[bot] wants to merge 0 commit into
Closed
fix(alertmanager): send resolved notifications to the AI-SRE relay [JDWLABS-476]#419jdwlabs-agent-bot[bot] wants to merge 0 commit into
jdwlabs-agent-bot[bot] wants to merge 0 commit into
Conversation
jdwlabs-agent-bot
Bot
requested review from
jdwillmsen and
jdwlabs-root
as code owners
September 5, 2026 03:18
jdwlabs-agent-bot
Bot
force-pushed
the
fix/JDWLABS-476-relay-send-resolved
branch
from
September 5, 2026 03:22
5a6791a to
739eb71
Compare
Member
|
Superseded by #420. The branch had to be reset and re-committed through the GitHub API so the commit carries GitHub's own signature; a locally signed commit under the bot author fails the signature check (reason: no_user). Resetting the branch to base made GitHub auto-close this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Flips
send_resolvedtotrueon theai-sreAlertmanager receiver so the relay receives resolved notifications, and records why in the adjacent comment. One line of config; thediscordreceiver in the same file already has it on.Why
The relay opens one Jira ticket per alert fingerprint and skips investigating repeats while that ticket is open. It never learned that a condition had cleared, so tickets stayed open until a person closed them, and every later firing of the same alert name was absorbed as a repeat. This week that left eleven stale tickets open, made the relay investigate nothing for 12h (
AiSreRelayInvestigatingNothing), and produced an auto-remediation PR proposing to disable dedup entirely.The relay-side feature that consumes this notification is jdwlabs/apps#213: note the resolve time on the ticket, close it after the alert stays resolved for a configurable grace (default 6h), cancel on re-fire, and never let a Done ticket suppress a new investigation. Merge order does not matter: the relay currently drops resolved alerts at the webhook, so enabling this first is inert until #213 deploys.
Verification
python3 -m unittest discover -s tools/tests -t tools/testspasses locally.tests/alertmanager-routing) is unaffected: receivers and routes are unchanged, only a delivery flag on an existing webhook config.Review gate
tenants/platform/is a CODEOWNERS-gated path, so this PR is authored under thejdwlabs-agent-botidentity to allow a human codeowner approval without the self-review wall.🤖 Generated with Claude Code
https://claude.ai/code/session_01FvYBM6o7wARm2v9jmDp1yY