Skip to content

webhook updates can be dropped after handler failure #32

Description

@Hqzdev

Problem

Webhook processing claims a Telegram update before the handler completes. If handleUpdate throws, the processed marker already exists in Redis, so a Telegram retry of the same webhook update is treated as duplicate and skipped.

Evidence

  • apps/bot/services/bot-gateway/src/bot.ts:206 calls claimUpdate(update.update_id) before handling the update.
  • apps/bot/services/bot-gateway/src/bot.ts:214 awaits handleUpdate(update) without a catch/release path.
  • apps/bot/services/bot-gateway/src/telegram-update-store.ts:43 stores the processed marker immediately with SET ... NX.
  • apps/bot/services/bot-gateway/src/telegram-update-store.ts:45 keeps that marker for 7 days.

Impact

A transient downstream error during webhook handling can permanently drop a Telegram update for the TTL window. This can lose successful payment updates, booking actions, or admin commands.

Expected Fix

Separate "in-progress claim" from "processed ack":

  • use a short-lived processing lock before handling;
  • mark an update as processed only after handleUpdate succeeds or after a deliberate terminal decision;
  • release/expire the processing lock on failure so Telegram retries can be handled;
  • add a regression test for webhook handler failure followed by retry.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

botBot gateway or bot runtimebugSomething isn't workingreliabilityReliability and failure-mode hardening

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions