Skip to content

varinfo: store evaluation outputs only in accumulators - #1500

Merged
sunxd3 merged 27 commits into
mainfrom
traces
Sep 30, 2026
Merged

sunxd3 merged 27 commits into
mainfrom
traces

Conversation

@yebai

@yebai yebai commented Sep 16, 2026

Copy link
Copy Markdown
Member

This replaces the value-owning VarInfo with an accumulator collection and removes the separate OnlyAccsVarInfo name. VarInfo() collects log densities without recording parameters. For example, VarInfo(VectorValueAccumulator()) opts into recording parameter values during evaluation; retrieve them with get_vector_values(vi) instead of vi.values. The VarInfo(rng, model) convenience constructor continues to collect both parameters and densities. This changes the public storage API.

Closes #1376.

Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Default and custom leaf contexts must not require a value accumulator
when the model has no latent inputs. Retain errors for absent latents.

Assisted-by: Codex <codex@openai.com>
@yebai
yebai added this pull request to stack #1505 September 16, 2026 21:16
@yebai
yebai requested a balanced review from Copilot September 16, 2026 21:17
@codecov

codecov Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.70886% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.34%. Comparing base (13dd27f) to head (5e60233).

Files with missing lines Patch % Lines
src/varinfo.jl 82.81% 11 Missing ⚠️
src/accumulators/priors.jl 25.00% 3 Missing ⚠️
src/test_utils/model_interface.jl 0.00% 3 Missing ⚠️
src/accumulators/vnt.jl 66.66% 2 Missing ⚠️
src/threadsafe.jl 91.30% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1500      +/-   ##
==========================================
+ Coverage   84.09%   84.34%   +0.25%     
==========================================
  Files          54       53       -1     
  Lines        4741     4638     -103     
==========================================
- Hits         3987     3912      -75     
+ Misses        754      726      -28     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown
Contributor

DynamicPPL.jl documentation for PR #1500 is available at:
https://TuringLang.github.io/DynamicPPL.jl/previews/PR1500/

Copilot stopped reviewing on behalf of yebai due to an error September 16, 2026 21:27
Docs previews, navbar updates, and preview cleanup share one branch.
Queue their runs together to prevent rejected pushes without dropping
pending previews for other pull requests.

Assisted-by: Codex <codex@openai.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Benchmarks @ 5e60233

Performance Ratio: gradient time divided by log-density time.

For very small models these ratios are noisy across runs and machines; raw primal and gradient timings are more reliable. The benchmarks are aimed at DynamicPPL developers and mainly catch obvious allocation or type-stability regressions. See benchmark notes for details.

===================================================================================================
                                               eval                       gradient                 
                                            ----------  -------------------------------------------
Model                        dim    linked      primal     FwdDiff    RvsDiff    Mooncake    Enzyme
---------------------------------------------------------------------------------------------------
Simple assume observe*         1     false     4.37 ns       10.69    1602.21       13.04      2.75
Simple assume observe*         1      true     4.38 ns       10.81    1657.82       13.10      2.74
Smorgasbord                  201     false     5.05 μs       76.07     143.21        6.55      3.91
Smorgasbord                  201      true      6.1 μs       75.81     156.71        6.77      2.26
Loop univariate 1k          1000     false     12.2 μs     1127.49     377.91        4.63      1.30
Loop univariate 1k          1000      true     18.1 μs     1279.04     264.35        3.18      0.82
Multivariate 1k             1000     false     1.51 μs     1357.40     605.74        1.61      5.48
Multivariate 1k             1000      true     1.48 μs     1485.69     639.14        1.72      5.62
Loop univariate 10k        10000     false    112.0 μs    31951.38     468.34        4.78      1.10
Loop univariate 10k        10000      true    167.0 μs    26870.14     323.16        3.25      0.76
Multivariate 10k           10000     false     15.8 μs    25756.63     588.85        1.87      4.03
Multivariate 10k           10000      true     15.3 μs    25078.29     634.44        1.76      4.19
Dynamic                      err     false      1.4 μs         err      68.82       19.63      7.29
Dynamic                       10      true     1.78 μs        1.87      75.43       16.72      9.45
Submodel*                      1     false     4.38 ns       11.51    1879.12       13.30     10.86
Submodel*                      1      true     4.38 ns       11.83    1914.98       13.08     10.73
LDA                            6      true      3.9 μs        1.36      14.17       39.54     41.21
===================================================================================================
Main @ 13dd27f
===================================================================================================
                                               eval                       gradient                 
                                            ----------  -------------------------------------------
Model                        dim    linked      primal     FwdDiff    RvsDiff    Mooncake    Enzyme
---------------------------------------------------------------------------------------------------
Simple assume observe*         1     false     5.56 ns        9.11    1326.76       10.75      8.21
Simple assume observe*         1      true     5.56 ns        9.55    1286.77       10.19      7.56
Smorgasbord                  201     false     5.39 μs       68.76     146.27        5.86      2.88
Smorgasbord                  201      true     6.44 μs       72.01     169.36        6.10      2.09
Loop univariate 1k          1000     false     16.9 μs      818.43     325.64        2.98      0.76
Loop univariate 1k          1000      true     18.3 μs     1316.48     303.45        2.80      0.73
Multivariate 1k             1000     false    906.0 ns     1612.44    1102.63        1.95      6.99
Multivariate 1k             1000      true    953.0 ns     1569.78    1092.17        2.03      7.46
Loop univariate 10k        10000     false    163.0 μs    22189.92     360.27        2.99      0.73
Loop univariate 10k        10000      true    178.0 μs    23844.82     334.28        2.84      0.68
Multivariate 10k           10000     false     8.56 μs    28293.08    1172.16        1.70      5.99
Multivariate 10k           10000      true     8.97 μs    26282.83    1148.59        1.67      6.13
Dynamic                       15     false     1.29 μs         err      60.13       15.10      5.92
Dynamic                       10      true     1.84 μs        1.61      63.34       12.69      6.91
Submodel*                      1     false     5.57 ns       10.06    1572.16       10.09      8.06
Submodel*                      1      true     5.57 ns        9.26    1496.11       10.73      8.04
LDA                            6      true     4.19 μs        1.38      11.19       31.54     28.94
===================================================================================================
Environment
Julia Version 1.13.1
Commit 96ca370cf0e (2026-09-25 19:34 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × INTEL(R) XEON(R) PLATINUM 8573C
  WORD_SIZE: 64
  LLVM: libLLVM-20.1.8 (ORCJIT, emeraldrapids)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 4 virtual cores)

@sunxd3 sunxd3 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for this. in general, the PR looks right to me. I think the only sampler in Turing still uses varinfo's value is EMCEE, which should be an easy migration.

Comment thread src/varinfo.jl
Comment thread src/transformed_values.jl
@yebai

yebai commented Sep 29, 2026

Copy link
Copy Markdown
Member Author

Thanks, Xianda. Both addressed on traces. A missing accumulator now throws an ArgumentError naming it (c85a691). On LinkAll(): the root cause was get_transform_strategy(vi) reconstructing the strategy from outputs, which #1469 rules out. I removed it along with link!!(vi, vns, model) (e654fbc); pass the strategy to init!! instead, e.g. LinkSome(Set(vns), UnlinkAll()). Inference also returns UnlinkAll() for empty values (f00ab3d). Agreed on Emcee.

@sunxd3

sunxd3 commented Sep 29, 2026

Copy link
Copy Markdown
Member

thanks!

On LinkAll(): the root cause was get_transform_strategy(vi) reconstructing the strategy from outputs, which #1469 rules out. I removed it along with link!!(vi, vns, model) (e654fbc); pass the strategy to init!! instead, e.g. LinkSome(Set(vns), UnlinkAll()).

makes sense, I notices the removal of partial links, but didn't connect to this.

@sunxd3

sunxd3 commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Should we merge to a preparation branch? I don't mind merging to main, just a reminder.
I misunderstood the stacked PR feature, pls ignore this comment

Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Claude Code <noreply@anthropic.com>
Assisted-by: Claude Code <noreply@anthropic.com>
Assisted-by: Codex <codex@openai.com>
For `loopm(1000)`, warmed `VarInfo(Xoshiro(1), model)` on Julia 1.13.0:
204.7 us / 12,017 allocations before; 20.2 us / 2,018 after.
Times are minima of 100 `@elapsed` calls; counts use `@allocations`.
Julia 1.10.12 after: 35.1 us / 2,010 allocations.

Single-accumulator mapping and setting allocate zero when warmed;
leave those paths unchanged. `VarInfo` and `init!!` inference checks and
both varinfo and accumulator suites pass on Julia 1.13.0 and 1.10.12.

Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
Assisted-by: Codex <codex@openai.com>
The observation method matched only `VNTAccumulator`, so the
`TSVNTAccumulator` used during threadsafe evaluation fell through to
the no-op fallback and pointwise log-likelihoods came back empty. This
also affected `setthreadsafe` models on main.

Assisted-by: Claude Code <noreply@anthropic.com>
Assisted-by: Claude Code <noreply@anthropic.com>
Comment thread docs/src/accs/values.md Outdated
Comment thread docs/src/accs/values.md Outdated
Comment thread src/abstract_varinfo.jl
Comment thread src/logdensityfunction.jl Outdated
Comment thread HISTORY.md Outdated
Assisted-by: Claude Code <noreply@anthropic.com>
Assisted-by: Claude Code <noreply@anthropic.com>
@sunxd3
sunxd3 added this pull request to the merge queue Sep 30, 2026
Merged via the queue into main with commit 6cb19e3 Sep 30, 2026
24 checks passed
@sunxd3
sunxd3 deleted the traces branch September 30, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rename OnlyAccsVarInfo to VarInfo

3 participants