Skip to content

feat!: graduate StateSpaceTimeSeries - #1113

Open
anevolbap wants to merge 11 commits into
pymc6_and_pymcmarketing1_migrationfrom
issue-758-ssts-graduate
Open

anevolbap wants to merge 11 commits into
pymc6_and_pymcmarketing1_migrationfrom
issue-758-ssts-graduate

Conversation

@anevolbap

@anevolbap anevolbap commented Jul 30, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #982, of #758 (P0) and of #696. Milestone 1.1.0. Targets pymc6_and_pymcmarketing1_migration for now and lands on main after 1.0.0rc1; rebased on the current migration tip after #1112 merged, so the diff is this PR's 11 commits only.

  • Remove the FutureWarning from StateSpaceTimeSeries.
  • BayesianBasisExpansionTimeSeries is unchanged. It stays experimental and keeps its FutureWarning, per review. A test pins that.
  • Edge-case guards and tests: short series, level_order below 1, seasonal_length below 2, missing values in y (the Kalman filter imputes them natively, which covers the P2 item of Feature parity with Google's CausalImpact #758 for this model), integer-index error path, and a warning when out-of-sample dates do not continue the training frequency. The missing-value test is correctness-marked so it runs against real NUTS: asserting that a posterior is finite says nothing against the suite's mocked pm.sample.
  • Two fixes to what the graduated model emits. It recovers the frequency of a regularly spaced training index, which rebuilding the index from raw values was dropping, so pymc-extras no longer warns about a missing frequency on every fit. And it builds the state-space model with verbose=False, so construction no longer prints the pymc-extras "Model Requirements" table, which tells the reader to declare priors this class declares itself.
  • New notebook interrupted-time-series-bsts.ipynb: trend and seasonality, then covariates, then spike-and-slab with inclusion probabilities. Added to gallery.yaml.
  • Document the smoothed-posterior R2 caveat on score().
  • Release note in a new .github/release-drafts/1.1.0.md, in the format of the rc1 draft, covering the removed warning and the new seasonal_length and level_order ValueErrors. The rc1 draft is untouched since this PR is not in that release.

Three items from #982 are deliberately not here, since each is a separate behavior change worth its own review: making StateSpaceTimeSeries the default model for InterruptedTimeSeries, exporting it at top level, and adding a cp.InterruptedTimeSeries(data, treatment_time) convenience entry point. The first depends on data-scaled priors (#935). The np.ndarray signature item listed in #982 is already stale: both time-series models take xr.DataArray today, and InterruptedTimeSeries dispatches on PyMCModel against RegressorMixin, not on the model class.

One gap worth naming: #758 and #982 both list "no seasonality" as an edge case to test, and no such path exists. build_model always adds a seasonal component, and pymc-extras ships no identity component, so the seasonal_length guard now says that plainly instead of pointing at a seasonality_component workaround that does not produce a seasonality-free model. A real trend-only option is a separate change if you want it.

Note for reviewers: score() is computed on smoothed in-sample predictions, so it reads optimistic relative to other models. Documented, not changed here.

On the notebook: it is honest about what the demo shows rather than overselling it. With 30 post-period days and diffuse default priors, the trend-only model does not identify the effect (95% interval includes zero, posterior probability of an increase around 0.75). Adding x1 and x2 tightens the interval substantially and raises that probability to about 0.87, but the effect is still not credibly non-zero at the 95% level. The plot titles report a Bayesian R² of 1 on the pre-period; the notebook explains that this comes from the Kalman smoother conditioning on the observed outcome and is not a measure of fit. The beta_exog posterior means sit on the true 2.0 and -1.5 while the individual 94% HDIs span zero, because the state-space level and the regressors compete for the same variation. In the selection example the two real predictors take the top two inclusion places but nothing clears 0.5, so the notebook tells the reader to use the ordering and not the level. If you would rather the notebook demonstrate a clean detection, the setup needs a change (shorter horizon, level_order=1, or data-scaled priors) and I will make it.

Verified locally with pymc 6.2.0, arviz 1.2.0, pymc-extras 0.14.0 on the rebased head: 2592 passed, 20 skipped, 6 deselected; the correctness lane 5 passed; diff-cover against pymc6_and_pymcmarketing1_migration covers 100% of the 10 changed executable lines; mypy clean; prek run --all-files clean; doctests for pymc_models.py 8 passed. The notebook calls .fit() on its three experiments and runs end to end under the mock runner (runner.py --notebook, 16 cells). Its stored outputs come from the earlier full run (runner.py --full, no mocks) and were not regenerated. The selection model reports rhat above 1.01, which is inherent to a spike-and-slab posterior rather than a sampling setup problem, so the notebook says so and points the reader at the inclusion probabilities; the covariate model shows 2 divergences. Sampling is not bit-reproducible across runs even with random_seed set, so the notebook prose avoids quoting values that drift.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@read-the-docs-community

read-the-docs-community Bot commented Jul 30, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 causalpy | 🛠️ Build #34808778 | 📁 Comparing bc57466 against latest (7e23946)

  🔍 Preview build  

854 files changed · + 146 added · ± 677 modified · - 31 deleted

+ Added

± Modified

- Deleted

@codecov

codecov Bot commented Jul 31, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.86364% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 97.15%. Comparing base (698c0e1) to head (bc57466).

Files with missing lines Patch % Lines
causalpy/pymc_models.py 90.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@                         Coverage Diff                         @@
##           pymc6_and_pymcmarketing1_migration    #1113   +/-   ##
===================================================================
  Coverage                               97.14%   97.15%           
===================================================================
  Files                                     136      136           
  Lines                                   24586    24672   +86     
  Branches                                 1403     1407    +4     
===================================================================
+ Hits                                    23885    23970   +85     
  Misses                                    484      484           
- Partials                                  217      218    +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch 2 times, most recently from 7b0298e to f82cde5 Compare July 31, 2026 03:49
@anevolbap
anevolbap force-pushed the issue-758-ssts-variable-selection branch from c58c9e3 to d158d16 Compare July 31, 2026 13:06
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch 2 times, most recently from 29a1d0c to d1c2cec Compare July 31, 2026 17:28
@anevolbap
anevolbap force-pushed the issue-758-ssts-variable-selection branch 2 times, most recently from b9da23f to 031c83d Compare August 3, 2026 20:19
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch from d1c2cec to 27c2dce Compare August 3, 2026 20:19
@anevolbap
anevolbap force-pushed the issue-758-ssts-variable-selection branch from 031c83d to e13d969 Compare August 4, 2026 02:36
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch from 27c2dce to 29c2147 Compare August 4, 2026 02:36
@anevolbap
anevolbap force-pushed the issue-758-ssts-variable-selection branch from e13d969 to e855435 Compare August 4, 2026 11:23
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch from 29c2147 to d598dd7 Compare August 4, 2026 11:23
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch from d598dd7 to 79b97a1 Compare August 28, 2026 17:52
@anevolbap
anevolbap force-pushed the issue-758-ssts-variable-selection branch from 29d9a10 to bdff957 Compare September 8, 2026 23:01
@anevolbap anevolbap closed this Sep 9, 2026
@anevolbap anevolbap reopened this Sep 9, 2026
@anevolbap
anevolbap removed this pull request from stack #1114 September 9, 2026 17:17
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch 2 times, most recently from aee603b to 7d3502c Compare September 10, 2026 01:52
anevolbap added a commit that referenced this pull request Sep 10, 2026
…stacklevel

get_inclusion_probabilities() and get_shrinkage_factors() returned a bare
RangeIndex, so the caller had to know the fit-time column order to read
them. That is the wrong contract for a feature whose output is a claim
about which covariates matter: the correctness test sliced positionally
and the notebook in #1113 relabelled the index by hand. Rows are now
indexed by the regressor names the model already tracks in _exog_names.

The beta_exog precedence warning goes to stacklevel 3. pm.Model's
metaclass calls __init__, so at 2 the warning was attributed to
pymc/model/core.py instead of the line that passed both arguments.

The vs_hyperparams docstring said the horseshoe scales its global
shrinkage from the data. Only the sample size comes from the data; the
residual scale in the Piironen and Vehtari rule is held at 1.
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch from 9ee29e1 to 91640f4 Compare September 10, 2026 21:03
@cetagostini

Copy link
Copy Markdown
Collaborator

indeed BayesianBasisExpansionTimeSeries should not be deprecated.

@cetagostini

Copy link
Copy Markdown
Collaborator

@anevolbap I don't recognize the branch, can we point to migration branch instead?

anevolbap added a commit that referenced this pull request Sep 18, 2026
…stacklevel

get_inclusion_probabilities() and get_shrinkage_factors() returned a bare
RangeIndex, so the caller had to know the fit-time column order to read
them. That is the wrong contract for a feature whose output is a claim
about which covariates matter: the correctness test sliced positionally
and the notebook in #1113 relabelled the index by hand. Rows are now
indexed by the regressor names the model already tracks in _exog_names.

The beta_exog precedence warning goes to stacklevel 3. pm.Model's
metaclass calls __init__, so at 2 the warning was attributed to
pymc/model/core.py instead of the line that passed both arguments.

The vs_hyperparams docstring said the horseshoe scales its global
shrinkage from the data. Only the sample size comes from the data; the
residual scale in the Piironen and Vehtari rule is held at 1.
@anevolbap
anevolbap force-pushed the issue-758-ssts-variable-selection branch from a897cec to cce4b29 Compare September 18, 2026 11:56
anevolbap added a commit that referenced this pull request Sep 18, 2026
Remove the experimental FutureWarning from StateSpaceTimeSeries: the API is now aligned with the other PyMCModel subclasses, covariates and variable selection are supported, and edge cases are guarded and tested.

This commit also pointed the BayesianBasisExpansionTimeSeries warning at StateSpaceTimeSeries as a deprecation. Review on #1113 asked to keep that model, so a later commit in this series restores it to its experimental state.

Record the change in the release notes under the 1.0.0 breaking-change section.

Completes the P0 graduation decision for #758.
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch from 18f6f7f to d02a04e Compare September 18, 2026 13:54
anevolbap added a commit that referenced this pull request Sep 18, 2026
…precating it

Review feedback on #1113: the model should not be deprecated. Restore its experimental FutureWarning and docstring to the base state, and update ARCHITECTURE.md, the ITS skill reference and the release notes. StateSpaceTimeSeries still graduates.
@anevolbap anevolbap changed the title feat!: graduate StateSpaceTimeSeries, deprecate BayesianBasisExpansionTimeSeries feat!: graduate StateSpaceTimeSeries Sep 18, 2026
@anevolbap

anevolbap commented Sep 18, 2026 •

Copy link
Copy Markdown
Collaborator Author

Dropped the deprecation: BayesianBasisExpansionTimeSeries is back to its base state (experimental, same FutureWarning), and only StateSpaceTimeSeries graduates. Title and body updated.

Also rebased onto the current #1112 head, so the conflict is gone, and the notebook now calls .fit() for the lazy lifecycle.

@anevolbap
anevolbap changed the base branch from issue-758-ssts-variable-selection to pymc6_and_pymcmarketing1_migration September 18, 2026 14:09
@cetagostini

Copy link
Copy Markdown
Collaborator

@anevolbap if we are ready then let me know moving this to ready for review, so I can merge.

@anevolbap
anevolbap marked this pull request as ready for review September 24, 2026 08:32
@drbenvincent drbenvincent added this to the 1.1.0 milestone Sep 27, 2026
@drbenvincent

Copy link
Copy Markdown
Collaborator

Moving this to the 1.1.0 milestone. The migration branch is being frozen so the PyMC 5 vs 6 baseline for #1048 can be pinned to the exact tree that goes to main for 1.0.0rc1, and this PR touches pymc_models.py. It can land on main after the rc.

Reject seasonal_length below 2 (previously an obscure ZeroDivisionError inside pymc-extras) and warn when out-of-sample dates do not continue the training frequency (forecast values map onto X's dates by position). Lock missing-value support with a test: the Kalman filter handles NaN in y natively and predictions stay finite, which covers the P2 item of #758 for this model.
Remove the experimental FutureWarning from StateSpaceTimeSeries: the API is now aligned with the other PyMCModel subclasses, covariates and variable selection are supported, and edge cases are guarded and tested.

This commit also pointed the BayesianBasisExpansionTimeSeries warning at StateSpaceTimeSeries as a deprecation. Review on #1113 asked to keep that model, so a later commit in this series restores it to its experimental state.

Record the change in the release notes under the 1.0.0 breaking-change section.

Completes the P0 graduation decision for #758.
New docs/source/notebooks/interrupted-time-series-bsts.ipynb covers the graduated StateSpaceTimeSeries with InterruptedTimeSeries: trend and seasonality only, exogenous control covariates, and spike-and-slab covariate selection with inclusion probabilities. Executed end to end. Cites the existing Brodersen et al. (2015) entry, adds the gallery card and the ITS toctree entry, updates the ARCHITECTURE.md model list, and documents the smoothed-posterior R2 caveat on score().

Part of #696 (the notebook, not yet the guidance on when BSTS is the better choice) and the documentation item of #758 P0.
level_order below 1 reached pymc-extras and failed there with 'index -1 is
out of bounds' or 'negative dimensions are not allowed', the same failure
mode the seasonal_length guard already covered.

The seasonal_length message offered a model without seasonality via a
custom seasonality_component. There is no such path: build_model always
adds a seasonal component, and pymc-extras ships no identity component.
… claim

mock_pymc_sample is session-scoped, so once any earlier test requests it
pm.sample stays patched. The three tests added here without it were
therefore mocked in any real run: test_missing_y_values_handled asserted
that a posterior is finite against prior draws.

It is now correctness-marked so it runs against real NUTS. The other two
assert shape and a warning, so they request the fixture explicitly, which
is what the rest of the file already does.
…able

Rebuilding a DatetimeIndex from raw values drops its freq, so every fit
emitted a pymc-extras 'No frequency was specific on the data's
DateTimeIndex' warning and the forecast index had to re-infer it. Recover
it when the observations are regularly spaced, and leave it unset when
they are not.

combined.build() also printed a rich 'Model Requirements' table to stdout
on every construction, telling the reader to declare priors that this
class declares itself a few lines later. The docstring example in #1112
had to wrap the call in redirect_stdout to hide it.
The thumbnail was force-added past .gitignore:18, which ignores that
directory because docs/source/conf.py regenerates it from the notebook on
every Sphinx build. It was the only tracked file there.

The skills reference still called both time-series models experimental.
The release note claimed the graduation put the class under the Tier 2
promise, but ARCHITECTURE.md defines Tier 2 mechanically and pymc_models
is already listed in docs/source/api/index.md, so the tier did not
change. Replaced with the two warnings that did stop firing.
Re-run against the current stack, so the stored outputs no longer carry
the pymc-extras requirements table or the frequency warning. That also
drops three absolute site-packages paths from the stderr output.

Three prose fixes. The plot title reports a Bayesian R2 of 1 with a
standard deviation around 1e-9, because the Kalman smoother conditions on
the observed outcome and interpolates the pre-period; the text now says
so instead of leaving the reader with an apparently perfect fit. The two
divergences in the covariates fit are acknowledged. The rhat paragraph
told the reader to fall back on the inclusion probabilities, which are
averaged over the very chains that disagree, so it now calls the ranking
provisional for that reason.

The hand-written index on the inclusion chart is gone: the labels come
from the model since #1112.
…precating it

Review feedback on #1113: the model should not be deprecated. Restore its experimental FutureWarning and docstring to the base state, and update ARCHITECTURE.md, the ITS skill reference and the release notes. StateSpaceTimeSeries still graduates.
The PR is in the 1.1.0 milestone, so its note leaves the 1.0.0rc1 draft and starts a 1.1.0 draft in the same format.
@anevolbap
anevolbap force-pushed the issue-758-ssts-graduate branch from d02a04e to bc57466 Compare September 28, 2026 17:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants