Skip to content

Add streaming AI card replies to the DingTalk channel - #538

Open
silence8013 wants to merge 4 commits into
scitix:mainfrom
silence8013:feature/dingtalk-ai-card
Open

silence8013 wants to merge 4 commits into
scitix:mainfrom
silence8013:feature/dingtalk-ai-card

Conversation

@silence8013

@silence8013 silence8013 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds streaming AI card replies to the DingTalk channel, so a DingTalk conversation shows a live-updating card instead of waiting for one final Markdown message. The card carries the streaming answer, high-level progress, and any agent-produced images, and it degrades to the existing sessionWebhook Markdown path whenever the card cannot be created or finalized.

Problem

DingTalk replies were a single Markdown message delivered after the whole turn finished. For a multi-minute investigation the chat stayed silent, and images the agent produced (charts, rendered visuals) arrived as separate messages detached from the answer. Lark already had a streaming card; DingTalk had no equivalent.

Solution

reply_mode=ai_card opts a channel into the card path:

  • dingtalk-card-api.ts wraps the three DingTalk card APIs: createAndDeliver to open the card, card/streaming for the answer body, and card/instances for structured content and the terminal atomic commit.
  • dingtalk-card-stream.ts drives updates through two independent single-flight, latest-wins senders (body vs. structured blocks) so a slow request can never be overtaken by a stale snapshot. finalize drains both before the last write.
  • dingtalk-card.ts defines the block contract consumed by the template: 0 answer, 1 progress, 2 tool status, 3 image. Raw tool arguments and results deliberately never reach the card — they may carry credentials or verbose infra output.
  • collectChannelResponse (shared by Lark and DingTalk, already imported by dingtalk.ts on main) gains two optional callbacks, onTextSnapshot and onToolActivity. Both are opt-in, wrapped in try/catch, and unset for Lark, so Lark behaviour is unchanged.
  • Images are uploaded before finalization so a successful upload becomes a type: 3 block in the same card; anything not embedded still falls back to separate image messages.

The template in docs/assets/dingtalk-ai-card-template.json renders the answer from the streaming content variable while the card is writing, and from the structured blockList once done. The writing state deliberately shows only type == 1 blocks: rendering type == 0 there made the final answer appear twice during the state transition, because the terminal update writes content, blockList, and flowStatus together and the client applies them with a visible window. Tool status blocks (type == 2) are hidden in both states — the bare tool names carried no signal for chat readers.

Alternatives rejected:

  • Rendering chart JSON or ```chart fences in the channel adapter. That would pull browser rendering into the Runtime process and duplicate render_chart, which already produces PNG artifacts inside the AgentBox isolation boundary.
  • Passing agent-authored image URLs to DingTalk. sanitizeMarkdownForDingTalk strips image URLs on purpose so a prompt-injected agent cannot make clients auto-fetch an attacker-controlled endpoint.

Test Plan

  • npx tsc --noEmit passes (no diagnostics)
  • npm test passes — could not complete locally; long-running commands are cut short in this environment. Targeted suites that cover the touched code all pass: src/gateway/channels 367 tests, src/core 646 tests, src/gateway/agentbox 287 tests. Please let CI be the gate on the full suite.
  • Manual verification — the card path needs a real DingTalk app: import the template, set reply_mode=ai_card plus card_template_id, then confirm streaming body, image blocks, and the terminal card state.

Rebase note for reviewers: this branch was replayed onto current main, which introduced sessionTurnLocks. cardController and the card timeline are now declared outside the turn-lock try because the card is finalized, and its timeline read for image embedding, after the lock is released. One behavioural consequence: the card is opened after the turn lock is acquired, so a busy session's "still working" notice goes out over the webhook instead of a card.

Architecture Checklist

  • Deployment mode: No resource sync, skill, or filesystem writes. Card API calls run in the trusted Runtime process in both local and K8s modes.
  • Security model: No new shell execution paths. Card bytes come from the agent's structured image blocks, not agent-authored URLs, and only pinned DingTalk OpenAPI hosts are contacted.
  • DB parity: No schema changes.
  • Tool protocol: No new tools.

General Checklist

  • Changes are focused on a single logical change
  • Commit messages follow Conventional Commits — these use the repository's imperative-summary style rather than a feat: prefix; happy to reword if that is required.
  • No unrelated changes included

jiyufeng048 added 4 commits September 1, 2026 11:24
The writing state rendered both the streaming `content` variable and every
blockList entry with `type < 3`. When finalization wrote the final answer,
the type 0 block and `content` landed in the same card data while the card
was still in the writing state, so the answer appeared twice until the
client switched to the done layout.

Restrict the writing timeline to progress blocks (`type == 1`) so the
streaming body stays the single answer source, and drop tool status blocks
(`type == 2`) from both states: the tool names carried no signal for chat
readers and only added noise. Runtime still emits type 2 blocks, so no
adapter change is needed.

Tested: re-parsed both JSON layers and asserted the four loop conditions;
vitest dingtalk, dingtalk-api, dingtalk-image (69 tests)
Not-tested: actual rendering in the DingTalk client after template import
Scope-risk: narrow
@silence8013

Copy link
Copy Markdown
Contributor Author
  1. portal channel
7f6d8ba49787ab04a40afcff067756fe 2. dingding talk image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant