Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ Model arguments now supply default observations. Use `decondition(model, :x)` in

`PrefixContext` has been removed. Use `prefix(model, vn; template)`; prefixes and nested storage templates now belong to the model.

`Context(rng, init_strategy, transform_strategy)` replaces `InitContext` and the context hierarchy. Pass it to `evaluate!!(model, context, outputs)`; custom initialisation and observation handling belong to strategies and accumulators.

# 0.42.13

Model bodies no longer contain a `try` block, so Libtask can tape them again.
Expand Down
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ BangBang = "0.4.1"
Bijectors = "0.16"
BridgeStan = "2"
Chairmarks = "1.3.1"
Compat = "4"
Compat = "4.10"
ComponentArrays = "0.15"
ConstructionBase = "1.5.4"
Distributions = "0.25"
Expand Down
15 changes: 14 additions & 1 deletion benchmarks/benchmarks.jl
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,18 @@ end
return (; x=x)
end

@model _indexed_observation(obs, mu) = obs ~ Normal(mu, 1)

"Indexed submodels with argument observations and one shared latent mean."
@model function indexed_submodels(obs)
mu ~ Normal()
x = similar(obs)
for i in eachindex(obs)
x[i] ~ to_submodel(_indexed_observation(obs[i], mu))
end
return (; mu=mu)
end

"Variables whose support varies under linking, or otherwise nontrivial bijectors."
@model function dynamic()
eta ~ truncated(Normal(); lower=0.0, upper=0.1)
Expand Down Expand Up @@ -142,7 +154,7 @@ function model_dimension(model, islinked)
DynamicPPL.init!!(
StableRNG(23),
model,
VarInfo(),
VarInfo(DynamicPPL.VectorValueAccumulator()),
DynamicPPL.InitFromPrior(),
transform_strategy(islinked),
),
Expand Down Expand Up @@ -321,6 +333,7 @@ function build_combinations(rng)
end
push!(models, ("Dynamic", dynamic()))
push!(models, ("Submodel", parent(randn(rng))))
push!(models, ("Indexed submodels 3k", indexed_submodels(randn(rng, 3_000))))
d = [1, 1, 1, 2, 2, 2]
w = [1, 2, 3, 2, 1, 1]
z = [1, 1, 2, 2, 1, 2]
Expand Down
4 changes: 2 additions & 2 deletions docs/src/accs/threadsafe.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ tilde-statement.

```@example 1
x = 1.0
context = DynamicPPL.InitContext(InitFromParams((; x=x)), UnlinkAll())
_, tsvi = DynamicPPL._evaluate!!(contextualize(model, context), tsvi)
context = DynamicPPL.Context(InitFromParams((; x=x)), UnlinkAll())
_, tsvi = DynamicPPL._evaluate!!(model, context, tsvi)
length(tsvi.accs_by_task)
```

Expand Down
6 changes: 3 additions & 3 deletions docs/src/accs/values.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ using Random: Xoshiro
return x[1:3] ~ Dirichlet(ones(3))
end
model = dirichlet()
context = InitContext(Xoshiro(1), InitFromPrior(), LinkAll())
context = Context(Xoshiro(1), InitFromPrior(), LinkAll())
_, vi = evaluate!!(model, context, VarInfo(VectorValueAccumulator()))
vector_values = get_vector_values(vi)
keys(vector_values)
Expand All @@ -41,7 +41,7 @@ A `RawValueAccumulator` records untransformed values. It does not retain stochas
block boundaries: indexed sites are represented by their individual indices.

```@example 1
context = InitContext(Xoshiro(1), InitFromPrior(), UnlinkAll())
context = Context(Xoshiro(1), InitFromPrior(), UnlinkAll())
_, vi = evaluate!!(model, context, VarInfo(RawValueAccumulator(false)))
raw_values = get_raw_values(vi)
keys(raw_values)
Expand All @@ -55,7 +55,7 @@ Raw values are used for chain construction. A whole variable such as
Reuse requires an explicit conversion outside evaluation:

```@example 1
context = InitContext(Xoshiro(1), InitFromParams(raw_values, nothing), LinkAll())
context = Context(Xoshiro(1), InitFromParams(raw_values, nothing), LinkAll())
retval, outputs = evaluate!!(model, context, VarInfo(VectorValueAccumulator()))
get_vector_values(outputs)
```
Expand Down
57 changes: 15 additions & 42 deletions docs/src/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,6 @@ Model
Model()
```

The context of a model can be set using [`contextualize`](@ref):

```@docs
contextualize
```

Some models require threadsafe evaluation (see [the Turing docs](https://turinglang.org/docs/usage/threadsafe-evaluation/) for more information on when this is necessary).
If this is the case, one must enable threadsafe evaluation for a model:

Expand Down Expand Up @@ -74,7 +68,7 @@ get_sample_input_vector
subsample
```

Internally, this is accomplished using [`init!!`](@ref) on:
Internally, this is accomplished using [`init!!`](@ref) with [`VarInfo`](@ref).

```@docs
to_vector_params
Expand Down Expand Up @@ -472,66 +466,45 @@ unflatten!!
internal_values_as_vector
```

### Evaluation Contexts
### Evaluation contexts

Internally, model evaluation is performed with [`AbstractPPL.evaluate!!`](@ref).

```@docs
AbstractPPL.evaluate!!
```

This method mutates the `varinfo` used for execution.
By default, it does not perform any actual sampling: it only evaluates the model using the values of the variables that are already in the `varinfo`.
If you wish to sample new values, see the section on [VarInfo initialisation](#VarInfo-initialisation) just below this.

The behaviour of a model execution can be changed with evaluation contexts, which are a field of the model.

All contexts are subtypes of `AbstractPPL.AbstractContext`.
Call `evaluate!!(model, context, varinfo)` to evaluate with an explicit context and collect
outputs in `varinfo`. Accumulators are reset before evaluation.

Contexts are split into two kinds:
The context is an evaluation input; it is not stored in the model.
Prefixes are stored separately from values. Conditioned and fixed values share one store, with each value carrying its role. Only latent sites reach the context; observations and tracked values go directly to accumulators.

**Leaf contexts**: These are the most important contexts as they ultimately decide how model evaluation proceeds.
For example, `DefaultContext` reuses values recorded by a `VectorValueAccumulator`, whereas `InitContext` obtains values either by sampling or from supplied parameters.
DynamicPPL has more leaf contexts which are used for internal purposes, but these are the two that are exported.
`Context` is the sole evaluation context. It supplies an RNG, an initialisation strategy,
and a transform strategy. The output `varinfo` never supplies latent inputs.

```@docs
DefaultContext
InitContext
DynamicPPL.Context
```

To implement a leaf context, subtype `AbstractPPL.AbstractContext` and implement `tilde_assume!!`.
Observations bypass the context and call `accumulate_observe!!` directly.
Customise value selection through `init` methods on initialisation strategies, and
observation handling through `accumulate_observe!!` methods on accumulators.

```@docs
tilde_assume!!
tilde_observe!!
DynamicPPL.store_coloneq_value!!
```

**Parent contexts**: These essentially act as 'modifiers' for leaf contexts.
Prefixes, conditioned values, and fixed values are stored on the model.

To implement a parent context, you have to subtype `DynamicPPL.AbstractParentContext`, and implement the `childcontext` and `setchildcontext` methods.
If needed, you can also implement `tilde_assume!!` for your context.
This is optional; the default implementation is to simply delegate to the child context.

```@docs
AbstractParentContext
childcontext
setchildcontext
```

Since contexts form a tree structure, these functions are automatically defined for manipulating context stacks.
They are mainly useful for modifying the fundamental behaviour (i.e. the leaf context), without affecting any of the modifiers (i.e. parent contexts).
Downstream evaluators that control execution directly can prepare arguments for `model.f`:

```@docs
leafcontext
setleafcontext
DynamicPPL.make_evaluate_args_and_kwargs
```

### VarInfo initialisation

The function `init!!` is used to initialise, or overwrite, values in a VarInfo.
It is really a thin wrapper around using `evaluate!!` with an `InitContext`.
The function `init!!` constructs a `Context` and evaluates the model, resetting the output accumulators.

```@docs
init!!
Expand Down
22 changes: 11 additions & 11 deletions docs/src/evaluation.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ The equivalent explicit-context call is:
```@example 1
using Random: Xoshiro

context = InitContext(Xoshiro(1), InitFromPrior(), UnlinkAll())
context = Context(Xoshiro(1), InitFromPrior(), UnlinkAll())
retval, accs = evaluate!!(model, context, VarInfo());
```

Expand All @@ -63,12 +63,12 @@ retval, accs = evaluate!!(model, context, VarInfo());
Evaluation separates the inputs that determine a model run from the outputs it records,
as proposed in [#1469](https://github.com/TuringLang/DynamicPPL.jl/issues/1469).

| Object | Responsibility |
|:------------- |:--------------------------------------------------------------------- |
| `Model` | Model function, arguments, and conditioned or fixed data |
| `InitContext` | RNG, initialisation strategy, and requested transform strategy |
| `VarInfo` | Output accumulators, with no separate parameter or transform storage |
| `retval` | The model body's ordinary Julia return value, distinct from its trace |
| Object | Responsibility |
|:--------- |:--------------------------------------------------------------------- |
| `Model` | Model function, arguments, and conditioned or fixed data |
| `Context` | RNG, initialisation strategy, and requested transform strategy |
| `VarInfo` | Output accumulators, with no separate parameter or transform storage |
| `retval` | The model body's ordinary Julia return value, distinct from its trace |

For a latent statement such as `x ~ Normal()`, the context's initialisation strategy
supplies `x`. Its transform strategy determines the transformed value and Jacobian.
Expand All @@ -81,9 +81,9 @@ use the same observation path. Fixed values are not scored, and tracked assignme
such as `z := x + y` are recorded when requested. None of these operations uses the
context to select a latent value.

The supplied leaf context replaces the model’s leaf context and is inherited by nested submodels.
The context belongs to the evaluation, not to `Model`, and is passed to nested submodels.
Inside a model body, `__context__` refers to this context; use `rand(__context__.rng, ...)`
for explicit random draws controlled by the evaluation's RNG. `init!!` constructs a `InitContext`
for explicit random draws controlled by the evaluation's RNG. `init!!` constructs a `Context`
and calls `evaluate!!`; custom value selection belongs in an initialisation strategy,
not a custom context type.

Expand All @@ -98,11 +98,11 @@ explicitly. For example, sample the model above, then evaluate it at the same pa

```@example 1
rng = Xoshiro(1)
context = InitContext(rng, InitFromPrior(), LinkAll())
context = Context(rng, InitFromPrior(), LinkAll())
retval, recorded = evaluate!!(model, context, VarInfo(RawValueAccumulator(false)))

params = get_raw_values(recorded)
context = InitContext(rng, InitFromParams(params, nothing), UnlinkAll())
context = Context(rng, InitFromParams(params, nothing), UnlinkAll())
repeated, scores = evaluate!!(model, context, VarInfo())

@assert repeated == retval
Expand Down
10 changes: 6 additions & 4 deletions docs/src/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,12 @@
or `VectorValueAccumulator` when those outputs are needed; `VarInfo(model)` remains
a convenience constructor that records vectorised values and log densities.

To reuse previous values, extract them explicitly before evaluating:
Replace `InitContext` with `Context`. `DefaultContext` and context subtyping are removed:
custom value selection belongs in initialisation strategies. To reuse previous values,
extract them explicitly before evaluating:

```julia
context = InitContext(rng, InitFromParams(get_vector_values(previous), nothing), LinkAll())
context = Context(rng, InitFromParams(get_vector_values(previous), nothing), LinkAll())
retval, outputs = evaluate!!(model, context, VarInfo())
```

Expand Down Expand Up @@ -110,9 +112,9 @@ vi = VarInfo(Xoshiro(468), model)
vals = [1.0, 1.0]
# Note this was `unflatten` (no exclamation mark) in the old code
vi = DynamicPPL.unflatten!!(vi, vals)
# Supply the inputs explicitly.
# Current syntax requires an explicit context.
_, vi = DynamicPPL.evaluate!!(
model, InitContext(InitFromParams(get_vector_values(vi), nothing), UnlinkAll()), vi
model, Context(InitFromParams(get_vector_values(vi), nothing), UnlinkAll()), vi
)
vi
```
Expand Down
6 changes: 2 additions & 4 deletions docs/src/onboarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,11 @@ Start with these docs:

### Prefer explicit evaluation state

Keep evaluation inputs in `InitContext` and choose output accumulators in `VarInfo`.
Keep evaluation inputs in `Context` and choose output accumulators in `VarInfo`.
For example, to reuse recorded parameters while collecting a different set of outputs:

```julia
context = InitContext(
rng, InitFromParams(get_vector_values(previous), nothing), UnlinkAll()
)
context = Context(rng, InitFromParams(get_vector_values(previous), nothing), UnlinkAll())
retval, outputs = evaluate!!(model, context, VarInfo(accumulators...))
```

Expand Down
4 changes: 2 additions & 2 deletions docs/src/tilde.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,11 @@ As described on the [Model evaluation page](./evaluation.md), there are three st
2. Transformation: figure out the untransformed (raw) value and the transformed value (where necessary); compute the relevant log-Jacobian.
3. Accumulation: pass all the relevant information to the accumulators, which individually decide what to do with it.

The method for `tilde_assume!!` (with `InitContext`) more or less implements this logic directly with three lines of code.
The method for `tilde_assume!!` (with `Context`) more or less implements this logic directly with three lines of code.
The implementation in `src/contexts/init.jl` follows this structure:

```julia
function DynamicPPL.tilde_assume!!(ctx::InitContext, dist, vn, template, vi)
function DynamicPPL.tilde_assume!!(ctx::Context, dist, vn, template, vi)
# 1. Initialisation
init_tval = DynamicPPL.init(ctx.rng, vn, dist, ctx.strategy)

Expand Down
13 changes: 1 addition & 12 deletions ext/DynamicPPLBridgeStanExt.jl
Original file line number Diff line number Diff line change
Expand Up @@ -524,7 +524,7 @@ function _stan_assume!!(distribution::StanDistribution, vn, template, vi, transf
end

function DynamicPPL.tilde_assume!!(
context::DynamicPPL.InitContext,
context::DynamicPPL.Context,
distribution::StanDistribution,
vn::DynamicPPL.VarName,
template,
Expand All @@ -535,17 +535,6 @@ function DynamicPPL.tilde_assume!!(
return _stan_assume!!(distribution, vn, template, vi, transformed_value)
end

function DynamicPPL.tilde_assume!!(
::DynamicPPL.DefaultContext,
distribution::StanDistribution,
vn::DynamicPPL.VarName,
template,
vi::DynamicPPL.AbstractVarInfo,
)
transformed_value = DynamicPPL.get_transformed_value(vi, vn)
return _stan_assume!!(distribution, vn, template, vi, transformed_value)
end

function _stan_constrain(transform::StanTransform, u::AbstractVector{<:Real})
_check_length(u, transform.input_dimension, "u")
return BridgeStan.param_constrain(transform.model, collect(Float64, u))
Expand Down
2 changes: 1 addition & 1 deletion ext/DynamicPPLInputProvenanceExt.jl
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ function check_input_provenance(rng, model, params)
defaults = map(_dualize_input, model.defaults)
values = DynamicPPL.map_values!!(_dualize_input, copy(model.values))
traced_model = DynamicPPL.Model{DynamicPPL.requires_threadsafe(model)}(
model.f, args, defaults, model.prefix, values, model.prefix_template, model.context
model.f, args, defaults, model.prefix, values, model.prefix_template
)
vi = DynamicPPL.VarInfo((InputProvenanceAccumulator(),))
strategy = DynamicPPL.InitFromParams(params, nothing)
Expand Down
17 changes: 5 additions & 12 deletions src/DynamicPPL.jl
Original file line number Diff line number Diff line change
Expand Up @@ -139,17 +139,8 @@ export AbstractVarInfo,
get_range_and_transform,
get_all_ranges_and_transforms,
get_logdensity_callable,
# Leaf contexts
AbstractContext,
contextualize,
DefaultContext,
InitContext,
# Parent contexts
AbstractParentContext,
childcontext,
setchildcontext,
leafcontext,
setleafcontext,
# Contexts
Context,
# Tilde pipeline
tilde_assume!!,
tilde_observe!!,
Expand Down Expand Up @@ -226,6 +217,8 @@ export AbstractVarInfo,
generated_quantities,
typed_identity

@compat public make_evaluate_args_and_kwargs, store_coloneq_value!!

# Reexport
using Distributions: loglikelihood
export loglikelihood
Expand All @@ -240,6 +233,7 @@ Abstract supertype for data structures that capture random variables when execut
probabilistic model and accumulate log densities such as the log likelihood or the
log joint probability of the model.

Implement `getaccs` and `setaccs!!` to provide an output container.
See also: [`VarInfo`](@ref).
"""
abstract type AbstractVarInfo <: AbstractModelTrace end
Expand All @@ -262,7 +256,6 @@ using .VarNamedTuples:

include("transformed_values.jl")
include("contexts.jl")
include("contexts/default.jl")
include("contexts/init.jl")
include("model.jl")
include("distribution_wrappers.jl")
Expand Down
3 changes: 2 additions & 1 deletion src/accumulators.jl
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,8 @@ function combine end
promote_for_threadsafe_eval(acc::AbstractAccumulator, ::Type{T}) where {T}

Convert `acc` to a new accumulator that works with threadsafe evaluation. The type parameter
`T` is the element type of the parameters that will be used for model evaluation.
`T` is a floating-capable type derived from the parameters, or `Any` when their type is
unknown. Preserve the accumulator's numeric type when `T` is `Any`.

See the docstring of `ThreadSafeVarInfo(vi, ::Type{T})` for more details.
"""
Expand Down
Loading
Loading