Skip to content

About

Papyri Unwrapping CLI: CT, density measurement, fibre tracing, winding field solve, mesh extraction, mesh flattening.

Topics

Resources

Stars

2 stars

Watchers

1 watching

Forks

Repository files navigation

winding-field-cli

Audit Coverage (local): 84.01% License: MIT Python 3.11–3.12 Release

Illustrated workflow: original CT → Lasagna surfaces + Qual V2 fibres/splines → winding field → 3D meshes → flattened charts

Build a winding field from scroll CT, then extract meshes and CT-textured surfaces. Defaults: Lasagna surface observations, Qual V2 fibres/splines and native surfaces flattened with Flatboi SLIM. Saved stages let you stop and continue later.

Start with 1 → 2 → 4: install, run the demo, then a small region. Use 3 for a full scroll only when its plan fits your resources; 5 resumes work, 6 exports surfaces, 7 reuses existing inputs, 8 summarizes options and results, and 9 links to the detailed guides.

1. Install

Install once with wf, then use wf in place of winding-field in every command of this README. Docker is the default: the image contains the complete CLI, Lasagna, native Flatboi (SLIM) and prebuilt accelerators, with no host Python or system packages to manage.

1A. Install with wf (default)

1. Get wf:

git clone https://github.com/LimeGS/winding-field-cli.git
echo "export PATH=\"$PWD/winding-field-cli/tools:\$PATH\"" >> ~/.bashrc && source ~/.bashrc

2. Install with Docker (needs Linux x86-64 and Docker with Compose; for GPUs also the NVIDIA Container Toolkit):

wf docker-install

This builds the image (GPU or CPU, chosen automatically) and checks it. Without Docker, run wf native-install instead: it installs on the host (Ubuntu 24.04, uses sudo for system packages) and the same wf commands keep working.

Options and details: CPU and CUDA containers.

1B. Manual host install (development)

Installs the CLI directly on the host. Both variants use the same dependencies and automatic CPU/CUDA selection. Prebuilt accelerators are not included, and on Linux x86_64 the pinned Flatboi (SLIM) runtime is downloaded and hash-verified on first use (--offline disables that download; see SLIM runtime).

wf native-install runs the quick install below for its checkout. If you install manually instead, run the commands in this README with winding-field in place of wf, and host paths in place of /data/... and /outputs/....

Quick install

On Ubuntu 24.04 with curl available, run this in Bash from the directory where you want the checkout:

source <(script=$(curl -fsSL https://raw.githubusercontent.com/LimeGS/winding-field-cli/main/tools/install.sh) && printf '%s\n' "$script" || printf 'return 1\n')

The installer uses sudo for system packages (unless root), clones into ./winding-field-cli, creates the CLI and isolated Lasagna environments, and runs doctor, then activates the CLI in this shell. Rerun from the same parent directory to reuse them. It does not pull updates, reset local edits, download models or create demo outputs; unrelated directories and incomplete environments are rejected rather than overwritten. Review the script before executing downloaded code, or use the manual steps below.

Manual install

Run these steps in order.

A. Clone the checkout and install shared system dependencies (Linux)

Ubuntu 24.04 (omit sudo when running as root):

sudo apt-get update
sudo apt-get install -y git
git clone https://github.com/LimeGS/winding-field-cli.git
cd winding-field-cli
sudo apt-get install -y $(cat tools/ubuntu-packages.txt)

B. Install the CLI and Lasagna — bootstrap with Python 3.12 below. CPU or CUDA is selected automatically from the available hardware and driver.

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip 'uv>=0.9.17'
bash tools/install-runtime.sh

The shared recipe installs both environments, including the plot and tifxyz Python extras, builds the CLI wheel and checks dependencies. The CLI remains in .venv; Lasagna uses its own Python 3.14.0 environment at ~/.cache/winding-field/lasagna-venv. Do not activate Lasagna: the CLI finds it automatically. Existing environments and additional user packages are retained; model weights download on first use, not during installation.

Docker (1A) uses this same recipe and Ubuntu package list, with explicit CPU/CUDA hash-locked dependency sets and exact Python versions. Host installation resolves from their shared dependency specifications with automatic backend selection; see the installation differences.

C. Check the installation — no model downloads:

winding-field doctor
winding-field demo --output runs/demo

The demo uses CUDA when available, otherwise CPU. Open runs/demo/index.html. Use a new output directory for each demo.

2. Run a demo

From the folder where you want your work (offline, synthetic data):

wf demo --output /outputs/demo

Open outputs/demo/index.html. wf creates data/ and outputs/ in that folder: put your inputs in data/ and read them as /data/...; results appear in outputs/ (/outputs/... in commands). Use a new output directory for each demo.

3. Run a winding field for a full scroll

Select the original CT, inspect storage/grid estimates, then run. Replace this public scan URL with your own URL or local 3D Zarr v2 path as needed. No --roi means the entire selected CT level. At native resolution, this scan's default preparation/product estimate is about 134 TB, even with a coarse field. A typical GPU server cannot run this example unchanged. Use the ROI in section 4 when the plan exceeds available disk; do not bypass the admission check.

CT='s3://vesuvius-challenge-open-data/PHerc0175A/volumes/20250521115057-8.640um-1.2m-116keV-masked.zarr'

wf plan --source "$CT" --voxel-um 8.64 \
  --field-spacing-um 300 --device cuda:0 --ct-mode stream \
  --until field --output /outputs/175a-full

wf run --source "$CT" --voxel-um 8.64 \
  --field-spacing-um 300 --device cuda:0 --ct-mode stream \
  --until field --output /outputs/175a-full
  • --until field runs Lasagna → Qual predictions/splines → field, then stops.
  • --field-spacing-um 300 chooses an example coarse solve grid; it does not reduce observation preparation or tracing scratch. Check plan before running.
  • --ct-mode stream fetches CT chunks on demand; products still need disk/RAM.
  • --device cuda:0 needs the CUDA image, which wf selects automatically on an NVIDIA host (or WF_VARIANT=cuda, see 1A). Use --device cpu in both commands for CPU. plan does not create the run; run --output must name a new directory.

Results: field/CHECKPOINT.json and field blocks, root FIELD.json with geometry/status, fibres/fibres.afv, and lasagna-source/output/ predictions.

For bounded preparation, add --bounded-preparation to both commands: recompute model windows, retain coarse observations and global fibre constraints, and gate field publication on global numerical verification. Requires --field-spacing-um, stride 1, and either native Qual V2 or imported AFV (--fibres together with --lasagna-input). ROI origins must be divisible by 4 for native Qual V2; imported AFV may use any origin. Add --fibre-downsample 4 to both commands for a 4× post-prediction fibre grid instead of the default 2× grid; ROI origins must then be divisible by 8. This uses about 8× fewer tracing voxels, not less native model inference or CT resolution, and can lose fine fibres. plan reports its physical grid separately. This trades retained arrays for recomputed context windows: even a small ROI can be much slower on CPU than the ordinary ROI recipe. Prefer CUDA when available; no whole-scroll throughput is established. Its endpoint-aligned field grid may have a slightly different pitch from the requested maximum; inspect plan.

Qual requires uint8, isotropic 8.64–9.6 µm observations. Set --voxel-um to the actual CT scale and choose --level/--stride accordingly. Field pitch is independent: any positive pitch fitting at least two nodes per axis is allowed; omitting it uses the observation grid, including its anisotropy. Unitless CT requires --voxel-um. Catalog selection uses --scroll ID --scan ID; ambiguous acquisitions need an exact scan or --source. --local /data/mirror.zarr adds a matching local chunk mirror to a remote source.

4. Run a field for a specific section of a scroll

ROI bounds are selected CT-level voxels, ZYX, with exclusive upper bounds. This example requests a 48³ region and a 17 µm field on CPU:

CT='s3://vesuvius-challenge-open-data/PHerc0175A/volumes/20250521115057-8.640um-1.2m-116keV-masked.zarr'
wf run --source "$CT" --voxel-um 8.64 \
  --roi 1488 1536 4032 4080 6048 6096 \
  --field-spacing-um 17 --device cpu --ct-mode stream \
  --until field --output /outputs/175a-roi

Replace the bounds for your region. Use plan instead of run to inspect grids first. Inference reads surrounding context; outputs remain inside the ROI. The field needs at least two nodes per axis.

Results: outputs/175a-roi/field/. To stop earlier, use --until prepare for predictions or --until fibres for traced/exported AFV. Continue with wf field /outputs/175a-roi.

5. Stop, inspect and resume a run

Point RUN at the complete run, not its field/ subdirectory. Press Ctrl-C or create STOP from another terminal, then wait for the process to exit:

RUN=/outputs/175a-roi
wf status "$RUN"
touch outputs/175a-roi/STOP   # the same run, seen from the host

Use RUN=/outputs/175a-full (and outputs/175a-full/STOP) instead if you admitted and started the full-scroll run.

Continue through field construction or through the selected terminal stage:

wf field "$RUN"
# Or:
wf resume "$RUN"

Both clear STOP, validate saved work and reuse completed stages. A run stopped at --until field reports PAUSED; inspect FIELD.json for the field result. Keep the original inputs, runtime, settings and run location. Changed inputs or implementation require a new output directory.

resume finishes native surfaces by default, original mapping for --flattening-method original, atlas for --surface-output atlas, or direct export for --surface-output tifxyz. It retains the saved flattening method; older runs without a method are not switched to SLIM. To extend an exhausted native time budget:

wf resume "$RUN" --extra-surface-seconds 600

Already published partial native levels are immutable; extra time does not remap them. Inspect status and coverage rather than treating a pause as completion.

If a --bounded-preparation field exhausted its sweeps without passing global verification, increase its absolute total budget (for example, from 4 to 8):

wf resume "$RUN" --field-sweeps 8

This extends the checkpoint budget without changing the original configuration or discarding progress. Repeating the target adds nothing; lower targets and changes to published fields are rejected. Verification still gates publication.

Opt-in automatic field convergence

Legacy commands above keep their current operators and limits. For a new automatic field, stop the initial workflow at --until fibres, then:

wf field-converge /outputs/prepared-inputs --output /outputs/field-auto \
  --max-sweeps 32 --global-rtol 1e-6
# After interruption, remove outputs/field-auto/STOP if present, then:
wf field-converge /outputs/field-auto --resume
# Only if that automatic generation exhausted its budget:
wf field-converge /outputs/field-auto --output /outputs/field-auto-next \
  --max-sweeps 64

This field-only command privately copies inputs without repeating preparation or fibres, fixes the global constraint rows at inception, and checks the stored field after complete sweeps. It stops on global convergence, finite total budget, stagnation or interruption; it never starts meshes/flattening automatically. Numerical convergence is not physical-sheet validation.

Existing legacy fields, including PHerc0483A, are not migrated or extended by this command. See criterion, safety, limits and examples.

Consultative field progress (development opt-in)

With an explicitly qualified observer-capable development core, add --field-advice to a new run/field-converge/prepared build request, then:

wf field-advice /outputs/my-run
# Prepared field roots are also accepted:
wf field-advice /outputs/prepared-field

The report never runs a solver or verification, stops work or adds sweeps. Published core 1.5.2 does not support this opt-in; new installations pin the stable core 1.7.0 from git at its exact commit, verified after install by commit id and source hashes (tools/verify-core.py) as recorded in provenance/core.json. See scope, recovery and interpretation.

6. Extract meshes and flattened CT from an existing field

Continue a CLI run

For the default native run from section 3 or 4, continue through rendering:

wf render /outputs/175a-roi

Use /outputs/175a-full for the full-scroll example. This automatically runs any preceding stages still needed and reuses completed work. Results are under the run's surfaces/: OBJ meshes in level-*/meshes/ and CT images, overviews and masks in level-*/component-*/. The saved run configuration selects the branch.

New native run/plan requests default to --flattening-method slim on Linux x86_64 (default elsewhere): each local chart keeps its projection/LSCM seed, then batches of charts are optimized by the pinned Flatboi SLIM, with the same geometry gates and seam assembly. Docker bundles Flatboi; host installs fetch and verify it on first use. Select --flattening-method default for the historical local mapping or original for global LSCM. Explicit slim is native-only; atlas/direct-field TIFXYZ retain their previous defaults. Add --level-workers N to compute N W levels in parallel processes (each needs its own memory; outputs equal a serial run). See native flattening.

Export a chosen W level as TIFXYZ

Use the run created in section 4 (or /outputs/175a-full from section 3). The installer already includes the TIFXYZ dependencies. No configuration files or repeated CT path/scale are needed:

wf export-sheets --run /outputs/175a-roi --levels -31 \
  --export-tifxyz --output /outputs/exports/W-31

Choose more levels with --levels -31 -30; these are raw field isovalues, as in the rest of the CLI. The command uses the saved field and CT metadata plus the original global LSCM recipe. Default XYZ coordinates are physical micrometres (unit 1, zero origin), not CT voxel indices.

The resolved field.json and flattening.json are saved automatically in the export directory and verified on resume; do not edit them. Repeat the same command with --resume to continue. Source runs remain unchanged. Missing or inconsistent saved metadata produces an error rather than guessed coordinates.

Main outputs:

outputs/exports/W-31/original/level-0000/
├── image.png
├── mask.png
└── tifxyz/{x,y,z}.tif

level-0000 means the first requested W (here −31), not W=0. See flattening setup and advanced reference for details.

Other export workflows

Task Command / guide
Choose LSCM/SLIM and patches/whole sheet export-sheets --flattening-solver lscm-slim|lscm|slim --scope patches|sheet — configuration
Extract and join continuous W ranges extract-range, join-ranges — recipe
Align neighbouring W levels on one UV grid export-uvw — reference map and settings
Prepare, flatten and resume a constant-W sheet constant-sheet — stages
Browse sheets and query pixel XYZ sheet-atlas build, sheet-atlas query — visual atlas
Continue PNG tiles from a saved patch map python -m winding_field.patch_continue — required inputs

All four solver/scope combinations preserve source IDs and support resumable CT/XYZ export. SLIM needs pinned Flatboi; LSCM does not. Do not mix these flags with historical --flattening-method recipes. UVW outputs are diagnostic: inspect coverage and deformation before using side cuts through the stack. export-sheets SLIM also requires a cgroup v2 memory/swap envelope; a regular unprivileged container without that host setup cannot run it. The native run renderer uses Flatboi without that envelope.

Inspect the result, not just the exit code. A completed workflow does not certify global convergence or physical sheet correctness. Native PARTIAL can mean that requested XYZ cells could not be formed; direct TIFXYZ BLOCKED publishes no READY/PARTIAL delivery. UVW_EMPTY_DIAGNOSTIC means no requested level retained any valid geometry. Black CT pixels can be real acquisition zeros: inspect masks and coverage rather than treating a saved image as a useful reconstruction.

All CT reads and sampling require the pinned vc_reader, installed with the CLI. Sampling uses verified native float64 interpolation; there is no backend setting. Resume older reader-bound runs with their original v1.5.0 runtime. See supported inputs and measured performance.

7. Custom input options

Fibres / splines (.afv)

An AFV from the same CT acquisition skips fibre inference/tracing. Lasagna still supplies surfaces. Set the ROI to match your data:

CT='s3://vesuvius-challenge-open-data/PHerc0175A/volumes/20250521115057-8.640um-1.2m-116keV-masked.zarr'
wf run --source "$CT" --voxel-um 8.64 \
  --roi 1488 1536 4032 4080 6048 6096 --field-spacing-um 17 \
  --fibres /data/fibres.afv --until field --output /outputs/from-fibres

CLI AFVs carry calibrated coordinates. Otherwise supply --fibres-geometry /data/geometry.json with CT identity, XYZ origin and spacing: descriptor. V2 names the model; its AFV container uses schema v1.

External field (.npy or block checkpoint)

For geometry-only extraction, copy and edit the prepared template:

cp examples/surface-run.example.json data/surface-run.json
# Replace paths, SHA-256 hashes, geometry, calibration and levels before running.
wf prepared surfaces --config /data/surface-run.json --output /outputs/external-surfaces

Paths resolve relative to the JSON (/data inside Docker). Remove field.mask if there is no support mask. The template's voxel_um: 8 makes spacing 2 equal to 16 µm; use voxel_um: 1 for micrometre origins/spacings. For chunk checkpoints and raw field-level calibration see prepared inputs.

Results: level-*.obj and renders/*.png under the output directory. These are geometry views; original CT/TIFXYZ needs the matching CT and the export-sheets recipe.

For several field levels, add --parallel-levels to prepared mesh or prepared surfaces. CPU, available RAM and disk headroom bound the number of independent processes automatically. Results are separate level-0000/, level-0001/, … bundles indexed by LEVELS.json, not a merged mesh. See parallel prepared levels.

External Lasagna predictions (.lasagna.json)

Use existing surface predictions from the same CT acquisition to skip Lasagna inference, its runtime and model download. For example, reuse a CLI export covering the selected ROI:

CT='s3://vesuvius-challenge-open-data/PHerc0175A/volumes/20250521115057-8.640um-1.2m-116keV-masked.zarr'

wf run --source "$CT" --voxel-um 8.64 \
  --roi 1488 1536 4032 4080 6048 6096 --field-spacing-um 17 \
  --lasagna-input /data/lasagna/surfaces.lasagna.json \
  --until field --output /outputs/from-predictions

The local Lasagna v2 manifest must reference scalar Zarr v2 cos, grad_mag, nx and ny channels. CLI exports include physical geometry; for uncalibrated predictions, add --lasagna-geometry /data/lasagna-geometry.json describing the manifest's base grid. Input format and geometry example.

Qual still supplies fibres by default. Add --fibres /data/fibres.afv (and its geometry descriptor if needed) to import both inputs without running either model. Keep the original prediction files for hash-verified resume.

With both --fibres and --lasagna-input, --stride 16 and --stride 32 are also supported (other routes retain 1..8). Stride samples the selected CT level from the ROI origin; it does not rescale imported fibre coordinates or Lasagna density. Planning and CT download visit only sampled CT chunks. Field pitch remains independent, e.g. --field-spacing-um 172.8, and both grids need at least two samples per axis. Use a new output to change stride; existing runs retain their original geometry and resume identity.

Local imports reuse bounded decoded/support caches and batch durable source ledgers and exports. Calibrated Lasagna observations go directly into the main maps, without a second normalized map copy. Add --import-workers 0 for resource-admitted automatic parallelism, or --import-workers N for a requested maximum; default 1 retains serial computation. Workers only compute: the parent validates source pins and writes results in input order. Remote inputs remain serial; unknown memory headroom also selects serial execution. This option is separate from --threads (PyTorch). Admission, recovery and benchmarking.

Without explicit or embedded physical geometry, local/HTTP/S3 manifests use the selected CT base-grid binding and pooled-cell sampling instead. This route also supports bounded preparation and resume RUN --offline; see the streaming input contract. Explicit/embedded physical-frame inputs retain calibrated interpolation.

8. Custom options

Defaults and configuration

New run/plan commands use --device auto: CUDA → MPS → CPU, admitting an accelerator only if the core's FP64 vector norm works. The concrete device is saved before execution; explicit devices and resumed runs are not reselected. MPS builds without FP64 support are skipped. There is no fallback after starting.

Flags or config files: all public commands accept --cli-config FILE in JSON or YAML (.yaml/.yml), using underscore option names. Explicit flags win over file values, then defaults. Native recipe options such as --config remain supported but are optional when their settings are supplied by flags. Scientific input arrays, hashes and frame declarations are still required; requests/checkpoints are generated automatically. See the complete flag/config mappings and flag-first Paris4 example.

Lasagna output reduction defaults to cos 2×, grad_mag/nx/ny 4× per axis. Use --lasagna-cos-downsample, --lasagna-grad-mag-downsample, --lasagna-nx-downsample, --lasagna-ny-downsample with 1, 2, 4 or 8 (nx/ny must match). Model input and CT texture stay unchanged; see Lasagna grids and recovery.

Options and results at a glance

Option Meaning
--level / --levels CT pyramid level / raw field isovalues (W × sheet_step)
--stride / --field-spacing-um Observation sampling / independent solve-grid pitch; bounded preparation uses endpoint-aligned spacing
--lasagna-input FILE / --lasagna-geometry FILE Import local surface predictions / explicit base-grid calibration; combinable with --bounded-preparation when --fibres supplies the fibre set
--surface-output native|atlas|tifxyz Native default / legacy charts and gallery / experimental direct-field export
--flattening-method slim|default Native local Flatboi SLIM (new-run default on Linux x86_64) / historical local mapping (default elsewhere)
--flattening-method original --flattening-config FILE Global LSCM with explicit output frame; independent of field resolution
--export-tifxyz Add mesh-derived XYZ TIFFs to native or original renders
--tifxyz-config FILE Separate request for the Linux-only experimental direct route
--coverage-target 1 --surface-time-limit 1800 Native mapped mesh-area target / cumulative active seconds per W
--level-workers 1 Native W levels computed in parallel processes; each needs its own memory; outputs equal a serial run
--import-workers 1 Local import workers (0 = resource-admitted automatic); results written in input order
--cache-gib 2 --reserve-gib 1 CT payload cache cap / free disk reserve, not RAM
--threads 4 PyTorch threads, not a global worker limit; idle threads sleep (OMP_WAIT_POLICY=PASSIVE) so concurrent runs share a host; set OMP_WAIT_POLICY=ACTIVE on a dedicated CPU host (~10% faster alone)
--device auto New-run default: usable CUDA → usable MPS → CPU; explicit PyTorch devices remain supported
--offline Disable CT networking; provision models/runtime separately
--invalid-value 0 Explicitly exclude acquisition fill; otherwise zero CT is valid
Output Where to look
Saved configuration, status and timings RUN.json
Field geometry and solver status FIELD.json, field/CHECKPOINT.json
Native results and progress SURFACES.json, surfaces/STATUS.json
Native meshes and CT surfaces/level-*/meshes/, surfaces/level-*/component-*/
Original global CT/masks/maps original/level-NNNN/, original/ORIGINAL.json
Atlas-only charts and gallery flat/, index.html
Direct-field publication tifxyz/TIFXYZ.json and READY.json or PARTIAL.json

For complete flags use command-specific help:

wf run --help
wf plan --help
wf resume --help
wf export-sheets --help
wf tifxyz --help
wf verify-tifxyz --help
9. Extra docs

Flattening · Constant-W visual atlas · Lasagna and AFV · Streaming · Prepared workflows · Surface tools · Attribution

Checks

The Audit badge follows installed-package CI pushes on main. Coverage is a local snapshot of this CLI package, not the numerical core's coverage or a live feed. On 2026-11-08, the installed-package suite on macOS arm64 / Python 3.12 passed 1,812 tests, with 41 skipped (Linux/CUDA and opt-in external controls). Statement coverage was 84.01%: 14,683 of 17,478 statements across 99 Python files, measured with coverage.py 7.17.2. This is not GPU/Docker qualification. To run the checks from the checkout:

uv pip install --constraint docker/cli.in '.[test,tifxyz]' pytest-cov
python -m pytest -q --cov=winding_field --cov-report=term-missing

End-to-end verification against the base branch (real CLI runs, flag and numerical diffs, cross-version resume; see REVIEW.md):

python tools/verify.py --baseline origin/main

Credits

Built on tools and open work from the Vesuvius Challenge community:

Thanks to their authors, maintainers and the community publishing scroll data and models. Exact source pins, adaptations and notices are in THIRD_PARTY.md and provenance/.

Related projects

Part of the LimeGS toolchain for the Vesuvius Challenge:

Citation

If you use this work, please cite it with the metadata in CITATION.cff (GitHub's Cite this repository button reads it).

License

MIT applies to repository-authored code. External tools, bundled upstream code, model weights and datasets retain their own licenses and terms; see attribution and notices.

About

Papyri Unwrapping CLI: CT, density measurement, fibre tracing, winding field solve, mesh extraction, mesh flattening.

Topics

Resources

Stars

2 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages