Skip to content

Correct the Bad COP two-part model decomposition - #488

Open
AliFozooni wants to merge 1 commit into
matheusfacure:masterfrom
AliFozooni:agent/fix-bad-cop-decomposition
Open

Correct the Bad COP two-part model decomposition#488
AliFozooni wants to merge 1 commit into
matheusfacure:masterfrom
AliFozooni:agent/fix-bad-cop-decomposition

Conversation

@AliFozooni

@AliFozooni AliFozooni commented Aug 15, 2026

Copy link
Copy Markdown

Summary

  • distinguish collider selection bias from mediator overadjustment and replace the predictiveness-only control rule
  • distinguish a valid two-part or hurdle model from an invalid causal interpretation of a conditional-on-positive contrast
  • define the participation probability p_t and positive-outcome mean m_t
  • show that recombining the margins gives E[Y_t] = p_t m_t and ATE = p_1 m_1 - p_0 m_0 under randomization
  • decompose the observed COP contrast into a causal quantity and a post-treatment selection term
  • show explicitly which participation margin is omitted by p_1(m_1-m_0)
  • explain the always-positive and treatment-induced principal strata in the existing illustrations, including monotonicity
  • clarify that randomization neither removes post-treatment selection nor determines its sign

Why this matters

Conditioning on a common effect opens a path and creates collider or selection bias. Conditioning on a mediator instead blocks part of the total effect and changes the estimand to a direct effect. The revised transition keeps these mechanisms separate and explains why a variable must be chosen from its causal role, not merely from how well it predicts treatment or outcome.

The previous Bad COP section also said that estimating participation and the positive outcome separately is biased. That is too broad: two-part models are valid for the marginal mean when their components are recombined. The actual problem is conditioning on the observed post-treatment event Y > 0 and interpreting m_1 - m_0 as a causal effect for one common population. The treated-positive group contains units with Y_1 > 0, while the control-positive group contains units with Y_0 > 0. Those are generally different principal-stratum mixtures even in a randomized experiment.

The rewrite preserves the warning while locating it precisely: the mistake is the causal interpretation of the selected contrast, not the two-part modeling strategy itself.

Validation

  • built the complete Jupyter Book successfully after both terminology and decomposition corrections
  • verified the rendered equations and all three existing illustrations
  • validated notebook JSON and checked the diff for whitespace errors
  • modified only markdown cells 40 and 42

The full build retains one unrelated pre-existing Chapter 21 lexer warning.

Closes #261
Closes #385
Closes #421
Closes #455

@AliFozooni
AliFozooni marked this pull request as ready for review August 15, 2026 17:59
@AliFozooni
AliFozooni force-pushed the agent/fix-bad-cop-decomposition branch from 42f05e8 to 20312eb Compare August 15, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chapter 7: Clarification Chapter 7 Issue on page /07-Beyond-Confounders.html Issue: Chapter 7, "Bad COP" section

1 participant