Skip to content

fix(brain): calibrate Metal from complete runtime allocations - #50

Merged
Draek2077 merged 1 commit into
mainfrom
fix/metal-calibration-budget
Sep 25, 2026
Merged

Draek2077 merged 1 commit into
mainfrom
fix/metal-calibration-budget

Conversation

@Draek2077

Copy link
Copy Markdown
Owner

What changed

  • Budget Metal models against the reported recommended working set with a 0.25 GiB reserve, and allow fits below 4096 context down to 1024.
  • Calibrate with two contexts that fit the estimated budget without first loading the saved hosting profile.
  • Temporarily request runtime verbosity 5 and capture complete Metal allocations, including the MTL0 memory breakdown that llama.cpp b11181 emits when the sample server closes. Reject incomplete lines rather than saving a partial measurement.
  • Show component costs and clearer budget labels in the model profile.

Evidence and limits

  • The Mac mini's 0.9.23 logs show MTL0 (Apple M4 Pro) self allocation of 15,714 MiB and a compute buffer of 136.2813 MiB. The existing parser only accepted older Metal ... buffer size = lines, so calibration rejected the load.
  • Seven focused test files pass (122 tests). Brain and app typechecks, lint, formatting, and the full pre-commit typecheck pass.
  • This candidate has not run on the Mac mini. A live calibration and inference trial is still needed before the next stable release.

Use the Metal working set as the budget basis and allow small-context calibration without first loading the saved hosting profile. Capture indexed MTL allocation tables through process close, require complete measurements, and expose the component costs behind admission decisions.
@Draek2077
Draek2077 merged commit b862ed7 into main Sep 25, 2026
18 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant