feat(web): render extracted images inline in the document reader#201
Open
KylinMountain wants to merge 2 commits into
Open
feat(web): render extracted images inline in the document reader#201KylinMountain wants to merge 2 commits into
KylinMountain wants to merge 2 commits into
Conversation
Follow-up to #197: long/short source docs embed `` references whose files were extracted at ingest but never displayed (they showed as a literal "!image"). Now they render inline in the reader. - Backend: GET /api/v1/document/image serves wiki/sources/images/** — narrowed to the images dir + raster suffixes with a traversal guard (read-only; cannot reach wiki pages/skills/arbitrary files). - MarkdownView: gain an optional image token, ENABLED ONLY when a resolveImageSrc prop is passed (the doc reader). Without it the `` token is omitted from the regex, so chat/wiki rendering is byte-for-byte unchanged. A new AuthedImage fetches the API path as a blob (the bearer token can't ride on <img src>; mirrors artifacts.ts) and revokes the object URL on unmount. - Doc reader: passes a resolver that normalizes both source path conventions (wiki-root `sources/images/...` and note-relative `images/...`) to the image endpoint; external/data URLs stay literal text. Images are now visible in the UI. (LLM answers are still image-blind — ingestion does no caption/OCR — which is a separate pipeline change.) Backend adds serve/traversal/suffix/404/auth tests (1251 passed). Verified in a browser: the Soochow Securities report's figures render inline.
- Image URL regex now tolerates one level of balanced parens, so an image whose path contains ')' (a legacy/cloud doc_name with ASCII parentheses, e.g. 'report (1)') is no longer truncated to a broken path + stray text [#1]. - The image endpoint sets an explicit media type per suffix instead of letting FileResponse guess: a .webp is served as image/webp even where mimetypes has no webp entry (would otherwise be text/plain → a blob-loaded <img> refuses to render it) [#2]. Adds a webp media-type test. - AuthedImage shows a muted dashed placeholder (the alt) on load failure instead of rendering nothing, so a missing image is visible rather than silently dropped [#3]. Verified: the balanced-paren regex extracts full paths for 'report (1)' and fullwidth-paren names; backend 1252 passed; frontend build green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Follow-up to the document reader (#199) — renders extracted images inline in the reader. Source docs embed
references whose files are extracted at ingest but were never displayed (they showed as a literal!image). Now they render as real images.Changes
GET /api/v1/document/imageserveswiki/sources/images/**. Narrowed to the images dir + raster suffixes (.png/.jpg/.jpeg/.gif/.webp, no SVG) with a containment/traversal guard, so this read-only sink can never serve wiki pages, skills, or arbitrary files.MarkdownView— gains an optional image token, enabled only when aresolveImageSrcprop is passed (the doc reader). Without it, thealternative is omitted from the regex, so chat and wiki-page rendering are byte-for-byte unchanged. A newAuthedImagefetches the API path as a blob (the bearer token can't ride on<img src>; mirrorsartifacts.ts) and revokes the object URL on unmount; it shows a placeholder while loading and renders nothing on failure.sources/images/…for long-doc JSON, note-relativeimages/…for short-doc MD) to the image endpoint. External /data:URLs stay literal text.Scope note
This makes images visible in the UI. LLM answers remain image-blind — ingestion does no caption/OCR (only extracts files + inserts placeholders), so figure/chart content still doesn't reach the model. That's a separate pipeline change, not attempted here.
Verification
1251 passed(adds serve / traversal / non-image-suffix / 404 / auth tests); ruff + mypy clean.npm run build(i18n + tsc + vite) green.!image) now render inline in the reader; chat/wiki rendering unaffected.