feat(tabpfn): add TabPFN-3 support with TabPFN-2 default by AzulGarza · Pull Request #25 · TimeCopilot/foundationforecast

AzulGarza · 2026-09-14T18:36:01Z

Summary

Upgrade tabpfn-time-series to >=1.2.0 and add model_path to TabPFN (TabPFN-2 default, TabPFN-3 opt-in)
Add real LOCAL integration tests (point, quantiles, level) for v2/v3 using TABPFN_TOKEN; keep mocked TabPFN in shared conftest for parametrized suite coverage
Add executed tabpfn-family.ipynb, docs updates, and CI wiring for TABPFN_TOKEN + TabPFN checkpoint cache

Test plan

pytest tests/models/test_tabpfn.py -m models (v2 passes; v3 skips until TabPFN-3 license accepted at ux.priorlabs.ai)
pytest tests/models/test_models.py -k tabpfn (mocked shared suite)
Executed docs/examples/tabpfn-family.ipynb (v2 results saved; v3 skipped in notebook when license not accepted)
Add TABPFN_TOKEN GitHub secret and accept TabPFN-3 license for full v3 CI coverage

Made with Cursor

Upgrade tabpfn-time-series for TabPFN-3 checkpoint support and the v1.2 predictor API; add python-dotenv as a dev dependency for test env loading. Co-authored-by: Cursor <cursoragent@cursor.com>

Expose model_path for checkpoint selection, default to TabPFN-2, and migrate to the tabpfn-time-series 1.2 predictor API with explicit tabpfn_config. Co-authored-by: Cursor <cursoragent@cursor.com>

Keep a mocked TabPFN in the shared models list for parametrized suite coverage; add dedicated LOCAL integration tests for v2/v3 point, quantile, and level forecasts using TABPFN_TOKEN from .env. Co-authored-by: Cursor <cursoragent@cursor.com>

Pass TABPFN_TOKEN from GitHub secrets and cache ~/.cache/tabpfn so LOCAL TabPFN tests reuse downloaded checkpoints across CI runs. Co-authored-by: Cursor <cursoragent@cursor.com>

Document TabPFN-2 default and TabPFN-3 opt-in, license requirements, and add an executed family comparison notebook on event pageview data. Co-authored-by: Cursor <cursoragent@cursor.com>

codspeed-hq · 2026-09-14T18:42:42Z

Merging this PR will improve performance by 13.04%

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 10 untouched benchmarks

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	Simulation	`test_maybe_infer_freq`	5 ms	4.4 ms	+13.04%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing feat/tabpfn-ts-3 (758af72) with main (cbc64bc)}

Include TabPFN-3 alongside TabPFN-2 in cross-validation outputs now that the NC license is accepted. Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot

🟡 Changes recommended

The review identifies a critical stale lockfile issue and additional API, mode-validation, and CI cache issues.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds opt-in TabPFN-3 support while retaining TabPFN-2 as the default, with dependency, test, documentation, and CI updates.

Changes:

Adds configurable checkpoint selection and updated TabPFN support.
Adds mocked and LOCAL v2/v3 tests.
Updates documentation, examples, and CI token/checkpoint caching.

File summaries

File	Reviewed changes and findings
`uv.lock`	Updates resolved dependencies.
`tests/models/test_tabpfn.py`	Adds TabPFN v2/v3 integration coverage.
`tests/models/conftest.py`	Adds shared mocked TabPFN coverage.
`README.md`	Documents TabPFN versions and licensing.
`pyproject.toml`	Critical (3 votes): `experiments/gift-eval/uv.lock` still records `tabpfn-time-series==1.0.3`; refresh it for frozen CI resolution.
`mkdocs.yml`	Adds the example to documentation navigation.
`foundationforecast/models/tabpfn.py`	Moderate (3 votes): Preserve existing positional parameter order or make `model_path` keyword-only. Moderate (2 votes): Validate or force LOCAL mode for TabPFN-3 on CPU.
`docs/model-hub.md`	Updates TabPFN model documentation.
`docs/examples/index.md`	Adds the TabPFN example.
`.github/workflows/ci.yaml`	Moderate (1 vote): Prevent the shared cache key from allowing Python 3.13 to save an empty cache; include the Python version or gate caching to TabPFN jobs.

Review details

Suppressed comments (1)

.github/workflows/ci.yaml:60

All matrix jobs use this same cache key, including Python 3.13 where TabPFN is excluded by the dependency marker. If that job saves first, it creates an empty cache and the <3.13 jobs' downloaded v2/v3 checkpoints cannot be saved under this key, so the cache is ineffective on subsequent runs. Include the matrix Python version in the key or gate this step to TabPFN jobs.

          key: ${{ runner.os }}-tabpfn-v1

Files reviewed: 9/11 changed files
Comments generated: 3
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  "pandas>=2.2.0 ; python_full_version >= '3.13'",
  "scipy",
-  "tabpfn-time-series==1.0.3 ; python_full_version < '3.13'",
+  "tabpfn-time-series>=1.2.0 ; python_full_version < '3.13'",



    def __init__(
        self,
+        model_path: str = TABPFN_V2_MODEL,


                CalendarFeature(),
                AutoSeasonalFeature(),
            ]
+        self.model_path = model_path


Document TabPFN-3 release notes and add a release checklist for PyPI publish. Co-authored-by: Cursor <cursoragent@cursor.com>

Move model_path to preserve positional API, validate TabPFN-3 LOCAL-only mode, refresh gift-eval lock, fix TabPFN cache key per Python version, sync uv.lock to 0.1.6, and move RELEASE.md out of docs for strict build. Co-authored-by: Cursor <cursoragent@cursor.com>

Add sample data loading and drop the forecast call so mktestdocs can execute the latest-changelog test without TABPFN_TOKEN or model downloads. Co-authored-by: Cursor <cursoragent@cursor.com>

Co-authored-by: Cursor <cursoragent@cursor.com>

AzulGarza and others added 5 commits September 14, 2026 12:35

chore(deps): bump tabpfn-time-series to >=1.2.0

9c20e7f

Upgrade tabpfn-time-series for TabPFN-3 checkpoint support and the v1.2 predictor API; add python-dotenv as a dev dependency for test env loading. Co-authored-by: Cursor <cursoragent@cursor.com>

feat(tabpfn): add TabPFN-3 support with TabPFN-2 default

a8e589e

Expose model_path for checkpoint selection, default to TabPFN-2, and migrate to the tabpfn-time-series 1.2 predictor API with explicit tabpfn_config. Co-authored-by: Cursor <cursoragent@cursor.com>

ci: add TABPFN_TOKEN and TabPFN checkpoint cache

50c4420

Pass TABPFN_TOKEN from GitHub secrets and cache ~/.cache/tabpfn so LOCAL TabPFN tests reuse downloaded checkpoints across CI runs. Co-authored-by: Cursor <cursoragent@cursor.com>

docs: add TabPFN family notebook and update documentation

0db65d2

Document TabPFN-2 default and TabPFN-3 opt-in, license requirements, and add an executed family comparison notebook on event pageview data. Co-authored-by: Cursor <cursoragent@cursor.com>

AzulGarza requested a lite review from Copilot September 14, 2026 18:45

Copilot started reviewing on behalf of AzulGarza September 14, 2026 18:45 View session

docs(tabpfn): re-run family notebook with TabPFN-3

2f0693d

Include TabPFN-3 alongside TabPFN-2 in cross-validation outputs now that the NC license is accepted. Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot AI reviewed Sep 14, 2026

View reviewed changes

AzulGarza and others added 4 commits September 14, 2026 12:55

chore(release): prepare v0.1.6 changelog and version bump

f70eb3f

Document TabPFN-3 release notes and add a release checklist for PyPI publish. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(docs): make v0.1.6 changelog example self-contained

d447f5b

Add sample data loading and drop the forecast call so mktestdocs can execute the latest-changelog test without TABPFN_TOKEN or model downloads. Co-authored-by: Cursor <cursoragent@cursor.com>

chore: remove RELEASE.md

758af72

Co-authored-by: Cursor <cursoragent@cursor.com>

AzulGarza merged commit b8eae67 into main Sep 14, 2026
11 checks passed

Navigation Menu

Uh oh!

feat(tabpfn): add TabPFN-3 support with TabPFN-2 default - #25

feat(tabpfn): add TabPFN-3 support with TabPFN-2 default#25
AzulGarza merged 10 commits into
mainfrom
feat/tabpfn-ts-3

AzulGarza commented Sep 14, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Sep 14, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

AzulGarza commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

codspeed-hq Bot commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 13.04%

Performance Changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AzulGarza commented Sep 14, 2026 •

edited

Loading

codspeed-hq Bot commented Sep 14, 2026 •

edited

Loading