Lock overlap IPW to the overlap population - #1207
Merged
drbenvincent merged 3 commits intoSep 26, 2026
Merged
drbenvincent merged 3 commits into
drbenvincent merged 3 commits into
Conversation
The overlap scheme is not the full-sample average treatment effect, and no numerical test asserted that. Co-authored-by: Cursor <cursoragent@cursor.com>
drbenvincent
commented
Sep 26, 2026
drbenvincent
left a comment
drbenvincent
left a comment
Collaborator
Author
There was a problem hiding this comment.
Review: PR 1207 vs issue #1200
Verdict: APPROVE (posted as --comment because GitHub disallows approving your own pull request).
Definition of done (issue 1200)
Add a regression test that _compute_ate_overlap matches the Hajek overlap contrast and does not match the full-sample average of the unit-level treatment effects. Do not change the estimator.
Overall assessment
The PR meets the definition of done.
- Only change is a new regression test (
causalpy/tests/test_ipw_overlap_identity.py); the estimator is untouched. - The two-stratum probe from the issue is reproduced (propensity 0.5 / effect 10 vs propensity 0.95 / effect 0).
- The test independently recomputes the Hajek overlap contrast
sum(w y | t) / sum(w | t) − sum(w y | c) / sum(w | c)withw = 1−e(treated) andw = e(control), and asserts agreement with_compute_ate_overlap(including the treated/control means) atatol=1e-12. - It also asserts the contrast differs from the full-sample mean of unit-level effects (
|ate − mean(y1−y0)| > 1), so a future change that silently switched to the full-sample ATE would fail. - This closes the gap called out in the issue:
test_ipw_get_ate.pyonly checkedate == trt − ntrt.
I ran the new test against the current estimator; it passed.
Findings
None. No correctness bugs, contract violations, or remaining DoD gaps.
Notes (non-blocking)
- Using
__new__to stub an unfitted IPW instance matches existing patterns in the suite (test_sdid_helpers.py,test_piecewise_its.py, etc.) and keeps the test free of MCMC. - Propensity values
0.5/0.95sit safely inside_prepare_ps’s clip interval, so the manual Hajek mirror is not distorted by clipping.
drbenvincent
marked this pull request as ready for review
September 26, 2026 14:21
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## pymc6_and_pymcmarketing1_migration #1207 +/- ##
===================================================================
Coverage 97.14% 97.15%
===================================================================
Files 130 131 +1
Lines 24033 24065 +32
Branches 1349 1349
===================================================================
+ Hits 23348 23380 +32
Misses 477 477
Partials 208 208 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
drbenvincent
added
tests
daimon-pymclabs
changed the base branch from
main
to
pymc6_and_pymcmarketing1_migration
September 26, 2026 16:01
…ap-population Bring the branch up to date after #1214 synced main into the migration branch.
drbenvincent
merged commit Sep 26, 2026
65e9202
into
pymc6_and_pymcmarketing1_migration
16 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1200
Summary
Test plan
pytest causalpy/tests/test_ipw_overlap_identity.py --no-covMade with Cursor