Skip to content

feat(images): 支持独立生图模型、批量生成与图片编辑 - #792

Merged
vastsa merged 9 commits into
vastsa:mainfrom
hui455:feat/image-generation
Sep 21, 2026
Merged

vastsa merged 9 commits into
vastsa:mainfrom
hui455:feat/image-generation

Conversation

@hui455

@hui455 hui455 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

问题与改动

新增独立的生图模型设置和默认 imagegen skill,在保留默认聊天模型的同时,让 Agent 通过 OpenAI-compatible Images 接口生成图片、编辑本地参考图或之前的生成结果。

每批最多 10 张、最多两个并发请求;生成与编辑结果保存为新文件,原图保留。复用现有文件预览能力,在对话中新增生图结果展示,收起工具详情后仍可见;未配置时可直接进入模型设置。默认模型与生图模型间距为 12px,服务商和模型字号一致。生图摘要行不提供更改或清除按钮;不可用时只显示“暂不可用”,切换通过服务商的模型高级设置完成。Windows 绝对路径的 Markdown 图片引用通过现有受限宿主读取接口显示,不扩大允许的 URL 协议。

宿主继续负责授权与凭据隔离,支持取消、逐项失败反馈、受限文件读写和下载校验,不自动重试收费请求。

设为生图的服务商+模型组合会从默认对话模型和聊天模型列表排除;其他服务商的同名模型不受影响。已有对话不改写历史,但若仍选中该生图绑定,发送前会被拦截并要求换对话模型。

另修复设置读写时的 infiniteProviderRetry is invalid:关闭或未配置状态规范化为 false,保存其他设置不再失败,也不会自动启用重试。

验证

  • JS / 桌面构建、桌面类型检查和 lint 通过。
  • 相关服务、路径、对话回归测试,以及隔离宿主 / HTTP / 存储、Chromium 和完整桌面 E2E 通过。
  • 真实 deepseek-v4-flash 对话调用 gpt-image-2.5:生成蓝色陶瓷杯,再引用原图编辑为绿色。两次图片请求成功,图片均为 1312×1199。
  • 修复 Markdown 显示后重新打开同一真实会话,两张图均可正常显示,没有重新调用生图接口。

范围与限制

本期不包含蒙版画笔、完整图片编辑器或其他生图协议。真实批量请求、其他服务商和 macOS/Linux 实机尚未验证;此前全量 Windows 测试仍有平台相关失败,不能将定向验证表述为全仓全绿。已合入 main b71fcf05a67d,并在候选 3cd5e3e5edd7 上通过构建、类型检查、lint、37 项定向测试及三组生图 E2E;远端 CI 状态另行核对。

PR #659 尚未合并,本期不做相关协调或复用,也不依赖它;待合并后再评估。本功能面向桌面内置聊天流程,不承担插件 SDK 的验收。

规格:docs/spec/03-runtime/21-image-generation.md。
设计说明:docs/adr/image-generation-capability.md。

Closes #790

已补齐 #659 评审中适用于本次范围的边界:本地工具异常的稳定错误码贯穿真实 sidecar RPC 与 ParentHostProxy;DALL·E 请求 b64_json,GPT Image 不发送 response_format;单图/双图/四图 multipart、超限引用拒绝、响应大小与错误体处理均有本地契约回归。这不等于新增官方服务在线实测。

截图

修改前:上游基线模型设置组件;浅色,1280×900,演示服务。

修改后:浅色设置组件,12px 间距;服务商和模型字号与默认模型一致,无更改/清除按钮。

高级设置组件:设为生图模型入口;浅色,1280×900。

最新完整桌面设置:生图摘要使用默认模型字号,无更改或清除按钮;此截图为隔离测试服务,浅色,1280×900。

真实接口生成蓝色陶瓷杯:DeepSeek 调用 gpt-image-2.5;在 Markdown 修复后重新打开保存的会话,没有重新生成。浅色,1280×900。

真实接口编辑为绿色杯子:引用上一张原图,新旧文件分别保存;在修复后的桌面重新加载。浅色,1280×900。

批量流程补充验证:完整桌面、本地模拟模型及图片服务,两张确定性测试图。浅色,1280×900。

配置恢复补充验证:完整桌面、本地测试服务,清除设置后提示并跳转,不发送图片请求。浅色,1280×900。

Let conversations create and refine image assets through a separate image model without replacing the chat default. Ship an imagegen skill and preserve generated files and per-item outcomes across reloads.

Authorize billable image calls through host-core, propagate cancellation, and bound requests, downloads, and local inputs. Cover configuration, batch failures, edits, permissions, and persistence with isolated tests.
Allow data images in the isolated UI fixture and wait for a decoded image rather than accepting an empty img element as preview coverage.
Align the image model summary with conversation defaults and keep image
results available when process details collapse. Resolve absolute image
references through the existing contained reader, including Windows paths.

Cover advanced selection, unavailable states and generation-to-edit
conversations with isolated desktop and renderer regression tests.
Keep the configured image provider/model pair out of conversation
defaults and Composer candidates. Reject retained image bindings before
inference without rewriting existing conversation history.

Verify provider-scoped filtering, binding replacement and the runtime
guard through selection tests and complete desktop interactions.
Normalize disabled or absent infinite retry to false so saving unrelated
settings does not fail boolean validation. Keep invalid writes rejected
and cover both renderer and main-process read-modify-write boundaries.
Carry stable local error codes through the sidecar RPC boundary while
preserving existing data and excluding arbitrary Error properties.
Request base64 responses for DALL-E without sending unsupported format
parameters to GPT Image models.

Cover the production proxy in a real child and multipart payloads through
a local HTTP endpoint, including bounded responses and reference limits.
Preserve upstream runtime and UI maintenance while validating the image feature against the current integration base. Retain both sets of E2E scenarios and clarify the existing settings recovery fixture.
Keep the new image capability discoverable in both documentation trees and satisfy locale, navigation and E2E contract checks. Clarify that configuration recovery opens Models settings.
…ests

The Composer readiness expression now checks the configured image binding. Pass the real shared predicate and settings into the existing expression harness so it evaluates the same dependencies as the component.

Cover provider-scoped image exclusion while preserving native capability and read-only behavior.

@vastsa vastsa left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

审查结论:未发现代码级阻塞问题。已核对权限授权、计划模式拒绝、模型绑定隔离、输入/下载边界、取消链路、渲染展示与持久化兼容性;定向运行时、宿主桥接、桌面回归和共享契约测试均通过。GitHub CI、文档、PR 基线和 Rust 门禁通过;Vercel 状态为部署授权失败,属于外部环境状态。批准合并。

@vastsa
vastsa merged commit 507fd45 into vastsa:main Sep 21, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] 桌面内置独立生图模型、批量生图与图片编辑

2 participants