Skip to content

Jupyter engine: emit real figures for image outputs (bd-5t6wvu7m)#412

Merged
cscheid merged 3 commits into
mainfrom
bugfix/bd-5t6wvu7m-jupyter-figure-outputs
Jul 24, 2026
Merged

Jupyter engine: emit real figures for image outputs (bd-5t6wvu7m)#412
cscheid merged 3 commits into
mainfrom
bugfix/bd-5t6wvu7m-jupyter-figure-outputs

Conversation

@cscheid

@cscheid cscheid commented Jul 24, 2026

Copy link
Copy Markdown
Member

Summary

A minimal jupyter doc (python cell with plt.plot([1,2,3])) produced no image in q2 render or q2 preview — only the text reprs. The kernel already sends the PNG (display_data with ['image/png', 'text/plain'], no MPLBACKEND forcing needed); q2 discarded it in format_outputs through two compounding bugs:

  1. Inverted MIME prioritytext/plain was checked first, so a matplotlib bundle (PNG + <Figure size …> text fallback) always took the text branch; the image branch was unreachable for typical figures. Now follows Q1's displayDataMimeType order scoped to what q2 handles: text/htmlimage/svg+xmlimage/pngimage/jpegtext/plain last.
  2. Placeholder image branch[Image output] instead of a file. The new FigureWriter writes <stem>_files/figure-html/cell-<i>-output-<j>.<ext> next to the source doc (Q1's naming; the layout JupyterEngine::intermediate_files already declares and knitr already produces), emits ![](…) inside the .cell-output-display div, and reports the <stem>_files dir via ExecuteResult::with_supporting_files — so q2 render copies figures to the output dir (bd-o8pr) and the preview capture embeds them for browser replay (bd-qbhp2cvv, Preview: transport engine-generated figures in the capture doc (bd-qbhp2cvv) #410). SVG payloads starting with <svg are written as text (Q1 rule); base64 decodes with nbformat multiline newlines stripped. Fail-soft: unwritable images fall back to the text/plain sibling.

Design decision for review: jupyter figure divs emit ::: {.cell-output-display} without the generic .cell-output class, converging on knitr's vendored figure hook (hooks.R:627). Q1's own engines disagree here (Q1-jupyter adds .cell-output to every output; Q1-knitr doesn't for figures); q2 resolves the asymmetry toward knitr so the cross-engine parity suite's strict semantic-class comparison holds with no figure-specific carve-out. The only generic .cell-output consumer (printStylesheet.ts page-break list) also matches img directly.

Test plan

  • TDD: the priority test (PNG + text/plain bundle must not render the text repr) was written first and observed failing. Six more unit tests cover file writing/naming, multi-output numbering, svg-as-text, html-beats-image, and the invalid-base64 fallback.
  • New parity_plot_output case in engine_output_parity.rs (per this strand's original scope): knitr plot(1) vs jupyter matplotlib must produce the same block shape, exactly one image each under doc_files/figure-html/, and captures reporting non-empty supporting_files. Skips without R/jupyter/matplotlib; verified actually running (~2s, real engines) and passing locally.
  • E2E (recorded in claude-notes/plans/2026-07-23-jupyter-figure-outputs.md): q2 render emits a real 556×413 PNG + <img>, manifest records the Engine/jupyter resource; q2 preview capture embeds the PNG and Chrome renders the plot via blob URL — output inspected.
  • cargo xtask verify passes end to end.

Discovered-from bd-qbhp2cvv (#410); closes the gap that blocked jupyter in that PR's E2E.

🤖 Generated with Claude Code

cscheid and others added 3 commits July 23, 2026 16:19
Two fixes in text_execute.rs format_outputs:

1. MIME priority now follows Q1's displayDataMimeType for HTML
   targets: text/html -> image/svg+xml -> image/png -> image/jpeg ->
   text/plain LAST. Previously text/plain was checked first, so a
   matplotlib display bundle (image/png + '<Figure size ...>' text
   fallback) always rendered as its text repr — the image branch was
   unreachable for typical figures.

2. The image branch writes real files instead of the '[Image output]'
   placeholder: new FigureWriter emits
   <stem>_files/figure-html/cell-<i>-output-<j>.<ext> next to the
   source doc (Q1's naming; the layout JupyterEngine::
   intermediate_files already declares and knitr already produces),
   references it as ![](...) inside the .cell-output-display div, and
   reports the <stem>_files dir via ExecuteResult::
   with_supporting_files — so q2 render copies figures to the output
   dir (bd-o8pr) and the preview capture embeds them for browser
   replay (bd-qbhp2cvv). SVG payloads starting with '<svg' are
   written as text (Q1 rule); base64 is decoded with nbformat
   multiline newlines stripped. Fail-soft: unwritable images fall
   back to the text/plain sibling, or the placeholder when none.

TDD: the priority test was written first and observed failing (text
repr emitted); six new unit tests cover file writing, naming,
multi-output numbering, svg-as-text, html-beats-image, and the
invalid-base64 fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…divs

jupyter figure divs now carry ONLY cell-output-display (no generic
cell-output), matching q2-knitr's vendored figure hook (hooks.R:627).
Q1's own engines disagree here (Q1 jupyter adds .cell-output to every
output; Q1 knitr does not for figures); q2 resolves the asymmetry
toward knitr so the parity suite's strict semantic-class comparison
holds with no figure-specific carve-out. The only generic
.cell-output consumer (printStylesheet.ts page-break list) also
matches img directly, so CSS behavior is unchanged.

New parity_plot_output case in engine_output_parity.rs: knitr plot(1)
vs jupyter matplotlib (trailing ; suppresses the execute_result so
the cells are semantically equivalent) must produce the same block
shape, exactly one image each under doc_files/figure-html/, and
captures reporting non-empty supporting_files. Skips when knitr,
jupyter, or matplotlib is unavailable; verified actually running
(2.1s, real engines) and passing locally.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the plan/E2E record (claude-notes/plans/2026-07-23-jupyter-
figure-outputs.md): q2 render emits a real 556x413 PNG +
<img src="pyfig_files/figure-html/cell-1-output-1.png">, manifest
records the Engine/jupyter resource; q2 preview capture embeds the
PNG (bd-qbhp2cvv transport) and Chrome renders the matplotlib plot
via blob URL — output inspected. Clippy: is_ok_and in the
matplotlib probe. cargo xtask verify passes end to end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@posit-snyk-bot

posit-snyk-bot commented Jul 24, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@cscheid
cscheid merged commit 03d1a5f into main Jul 24, 2026
8 checks passed
@cscheid
cscheid deleted the bugfix/bd-5t6wvu7m-jupyter-figure-outputs branch July 24, 2026 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants