Skip to content

Label mesh rendering, skeleton shaders, cross-section cameras, and recolor crossfades - #4

Open
davidackerman wants to merge 25 commits into
mainfrom
feature/mesh-and-slice-rendering
Open

Label mesh rendering, skeleton shaders, cross-section cameras, and recolor crossfades#4
davidackerman wants to merge 25 commits into
mainfrom
feature/mesh-and-slice-rendering

Conversation

@davidackerman

@davidackerman davidackerman commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

@stuarteberg

Summary

Continuation of the label-mesh-rendering branch (renamed from feature/label-mesh-rendering, previously PR #3, closed when the branch was renamed). Adds:

  • Blockwise label meshing with decimation/smoothing, configurable local data roots, and per-keyframe background — the original label-mesh-rendering work.
  • A fly-through waypoint toggle for camera keyframes.
  • A skeleton shader expression colorizer: evaluates the NG GLSL skeleton-shader subset (uicontrol floats, prop_*() attributes, math/vec* functions, emitRGB/RGBA) so tube colors match custom NG colormaps, not just the older piecewise-smoothstep pattern.
  • Orthographic camera support for Neuroglancer 2D/cross-section keyframes: converts captured 2D panel state (position, cross-section plane, pixel-accurate ortho scale from the captured viewport) into a Blender orthographic camera, threads projection/ortho scale through camera interpolation for 2D-to-2D transitions, and upgrades older saved 2D keyframes at render time without mutating the project file.
  • Slice contrast fixes: display window now scales from the source dtype instead of the crop's observed max, bare shaderControls.range (no window) is honored, and contrast limits now survive interpolation between keyframes.
  • Recolor crossfade: meshes are matched by layer name and split by segment membership so segments common to two keyframes stay continuously visible instead of blinking when the selected id set changes size; same-source layers that recolor overlapping segments mid-transition render as one merged mesh with two baked vertex-color sets crossfaded by a material mix factor, instead of stacking independently-fading duplicate meshes (which could fail to resolve in Cycles and hide geometry).
  • ER mesh slice/cut comparison spikes (exploratory, spikes/).

Label meshing (mesh_from_labels):
- Read + mesh the ROI block-by-block (parallel) and weld seams, so a sparse
  but huge bbox doesn't materialize the whole box at once. Auto-engages by
  planned read size vs. a memory threshold that respects the job's cgroup
  limit (LSF/containers), not just the node's free RAM.
- target_vertices caps what we LOAD (planner never meshes finer than the
  budget); decimate_fraction then reduces below it, per segment so instance
  colors survive. fast-simplification added as the decimation backend.
- Removed the confusing simplify_factor knob entirely (smooth + decimate only).

Per-keyframe 3D background:
- Capture Neuroglancer projectionBackgroundColor per keyframe on bake/update,
  interpolate it across transitions, and apply/animate it per frame in both
  the PNG render and the .blend export.
- Propagate it across keyframes via the "Update current frame" prompt, mirroring
  the change back into each keyframe's stored ng_state so the NG link round-trips.

UI: render controls regrouped into labeled rows (movie/render/mesh/scene) and
the timeline bar into move/keyframe/output; "quality"->"detail"; decimate
dropdown; opacity slider relabeled (all layers). README + tests updated.
Per-keyframe `waypoint` flag: the camera passes through the keyframe at speed
instead of easing to a stop (and any hold is ignored), so a run of waypoints
reads as one continuous flow. Easing is derived per-segment from endpoint type —
stop->stop ease-in-out, stop->wp ease-in, wp->stop ease-out, wp->wp linear — which
also generalizes the director's smooth-ends curve. Timelines with no waypoints are
unchanged. Toggle lives in the transition-edit popover; persisted via DurationReq.
Evaluate the NG skeleton shader subset (uicontrol floats, prop_*()
attributes, math/vec* functions, emitRGB/RGBA) directly so tube colors
match custom NG colormaps, not just the older piecewise-smoothstep
pattern. Threads shader text into the mesh geometry cache key so a
shader edit invalidates baked tube colors.
…amera

NG's 2D panels (xy/xz/yz and oblique cross-sections) are orthographic,
not the 3D panel's perspective view. Convert the captured NG state into
an orthographic Blender camera (position, cross-section plane, and a
pixel-accurate ortho scale derived from the captured viewport size),
thread projection/ortho_scale_nm through camera interpolation so 2D-to-
2D transitions behave like NG movie states, and upgrade older saved 2D
keyframes at render time without mutating the project file.

Also fixes slice contrast: the display window is now scaled from the
source dtype instead of the crop's observed max (a dark/bright crop no
longer skews brighter/darker than NG), a bare shaderControls "range"
(no "window") is honored, and contrast limits now survive interpolation
between keyframes instead of being dropped.
Meshes are now matched by layer name and split by segment membership:
segments common to both keyframes stay continuously visible, removed
ids fade out, and added ids fade in. This fixes common segments
blinking when a layer's selected id set changes size across a
transition.

When several same-source layers recolor overlapping segments mid-
transition, render them as one merged mesh with two baked vertex-color
sets (Col/Col2) crossfaded by a material mix factor, instead of
stacking independently-fading duplicate meshes — coincident
transparent surfaces could fail to resolve in Cycles and hide geometry
behind them for the whole crossfade.
Exploratory scripts comparing zmesh vs. voxel-surface vs. blockwise-
decimated ER meshes and clip/cap methods, following the repo's
existing spikes/ convention for throwaway experiments.
203MB of binary PLY/PNG comparison output shouldn't be tracked.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant