diff --git a/causal-inference-for-the-brave-and-true/Conformal-Inference-for-Synthetic-Control.ipynb b/causal-inference-for-the-brave-and-true/Conformal-Inference-for-Synthetic-Control.ipynb index 89932ae..ae453ff 100644 --- a/causal-inference-for-the-brave-and-true/Conformal-Inference-for-Synthetic-Control.ipynb +++ b/causal-inference-for-the-brave-and-true/Conformal-Inference-for-Synthetic-Control.ipynb @@ -421,9 +421,11 @@ "source": [ "If we postulate the null of no effect, the data under that null means that $Y(0) = Y(1) = Y$, which is just the trajectory of observed outcome we see for the treated state of California. Now, if we postulate that the null is -4, that is, Proposition 99 decreases cigarette sales by 4 packs, then $Y(0) = Y(1) - (-4)$, which shifts the trajectory of the post treatment outcomes by +4. This is very intuitive. If we think the bill decreases cigarette sales, then, in the absence of it, we should see higher levels of cigarette sales than the one we have in our observed data. \n", " \n", - "The next part of the inference procedure is to fit a model for the counterfactual $Y(0)$ (which we get with the function we just created) in the entire data, pre **and** post-treatment period. This is an important distinction between how we usually fit synthetic controls. The idea here is that the model must be estimated with the entire data, under the postulated null hypothesis, to avoid huge post intervention residuals. With this model, we then compute the residuals $\\hat{u_t} = Y_t - \\hat{Y}_t(0)$ for all time periods $t$.\n", + "The next step deliberately fits the counterfactual model on the entire null-adjusted series, including the pre- **and** post-treatment periods. Under the sharp null $H_0:\\theta=\\theta^0$, subtracting $\\theta_t^0$ imputes the otherwise missing untreated outcome in every post-treatment period: $Y_t(0)=Y_t-\\theta_t^0$. We therefore have a complete candidate path for $Y(0)$ under that null, can refit the model on all periods, and can compute $\\hat{u}_t=Y_t(0)-\\hat{Y}_t(0)$ for every $t$. This is the null-imputation procedure in [Chernozhukov, Wüthrich, and Zhu (2021)](https://doi.org/10.1080/01621459.2021.1920957); it is not a device for preventing large post-treatment residuals. Large residuals in the designated post-treatment block are precisely the evidence against a candidate null that the test statistic is meant to detect.\n", " \n", - "The function to do that first uses the `with_effect` function we created earlier to generate data under then null. Then, it fits the model in this data under the null. Next, we estimate $Y(0)$ by making predictions with the recently fit model. Finally, we compute the residuals $\\hat{u}_t$ and stores everything in a dataframe." + "Why include the imputed post-treatment observations in the fit? The cyclic block permutations later compare the statistic for the designated post-treatment block with the same statistic after that block position is reassigned throughout the residual sequence. Fitting once on the complete series under the null puts the original block and its cyclic reassignments on the same footing. A pre-treatment-only fit would instead mix in-sample residuals before the intervention with out-of-sample residuals after it; a permutation could then detect that change in fitting status rather than a structural break. That would define a different calibration, not a drop-in replacement for this procedure.\n", + " \n", + "The function below first uses `with_effect` to construct the complete series under the candidate null. It then fits the model to that null-adjusted series, predicts $Y(0)$, computes $\\hat{u}_t$, and stores the residuals together with the post-treatment indicator." ] }, { @@ -683,9 +685,9 @@ "\\text{P-value} = \\frac{1}{|\\Pi|}\\sum_{\\pi \\in \\Pi} \\mathcal{1}\\{S(\\hat{u}_{\\pi_0}) \\leq S(\\hat{u}_{\\pi})\\}\n", "$$\n", " \n", - "and $\\hat{u}_{\\pi_0}$ is the original (unpermuted) vector or residuals. In plain terms, we are simply finding the proportion of times that the unpermuted test statistic is higher (more extreme) than the test statistics obtained by all possible block permutations. \n", + "and $\\hat{u}_{\\pi_0}$ is the original (unpermuted) vector of residuals. In plain terms, we find the proportion of block permutations whose test statistic is at least as large (as extreme) as the original unpermuted statistic. This direction matches the indicator above: $S(\\hat u_{\\pi}) \\geq S(\\hat u_{\\pi_0})$. \n", " \n", - "To implement this, we will make use of the `np.roll` function, which takes an array and circles it, mujustch like we've represented in the image above." + "To implement this, we will make use of the `np.roll` function, which takes an array and cycles it, much like we've represented in the image above." ] }, {