Skip to content

Commit 13a2374

Browse files
igerberclaude
andcommitted
docs(firpo-possebom review): sharpen CI test-inversion boundary; drop disputed license tag
Address local codex review (round 1): - P1: make the confidence-set inversion boundary explicit — keep grid values with the strict p^f > γ (Eqs 14/16/18), excluding p^f = γ even though the rejection rule (p^f < γ) does not reject it; the two boundaries differ at the discrete permutation p-value γ (a multiple of 1/(J+1)). - P2: drop the disputed CC-BY-NC license tag (publisher metadata, not paper-sourced; aggregators disagree CC-BY-NC vs CC-BY-NC-ND) — keep the unambiguous "open access", matching house style (abadie-2021 / chen-santanna-xie). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 9db58eb commit 13a2374

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

docs/methodology/papers/firpo-possebom-2018-review.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
**Authors:** Sergio Firpo (Insper), Vitor Possebom (Yale University)
44
**Citation:** Firpo, S., & Possebom, V. (2018). "Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets." *Journal of Causal Inference*, 6(2), 20160026.
5-
**PDF reviewed:** https://doi.org/10.1515/jci-2016-0026 (published *Journal of Causal Inference* version, CC-BY-NC; received 15 Nov 2016, revised 6 Aug 2018, accepted 11 Aug 2018, 26 pp). Per the project's PDFs-never-committed convention the local PDF is kept outside the repository; the published J. Causal Inference version (DOI 10.1515/jci-2016-0026) is the authoritative source. All equation, section, and footnote numbers below are pinned to that version.
5+
**PDF reviewed:** https://doi.org/10.1515/jci-2016-0026 (published *Journal of Causal Inference* version, open access; received 15 Nov 2016, revised 6 Aug 2018, accepted 11 Aug 2018, 26 pp). Per the project's PDFs-never-committed convention the local PDF is kept outside the repository; the published J. Causal Inference version (DOI 10.1515/jci-2016-0026) is the authoritative source. All equation, section, and footnote numbers below are pinned to that version.
66
**Review date:** 2026-06-01
77

88
> Scope note: this paper extends the **permutation / placebo inference** procedure of Abadie, Diamond & Hainmueller (the SCM benchmark) in two ways — (1) a **sensitivity analysis** that parametrically re-weights the placebo p-value away from the equal-weights benchmark, and (2) testing **any sharp null hypothesis** (not only "no effect whatsoever") via a modified RMSPE statistic, which it **inverts to construct confidence sets** for the treatment-effect path. It also generalizes to arbitrary test statistics, multiple outcomes (familywise error control), and multiple treated units (a pooled effect). This review is the **Step-1 fidelity artifact** for a forthcoming SCM **confidence-set / CI-by-test-inversion** implementation (PR-B) layered on the existing `SyntheticControl` estimator; the sensitivity-analysis and multiple-outcome / multiple-treated extensions are documented here but flagged **deferred**. The estimator itself (donor weights `W`, predictor importance `V`) is taken as given from ADH 2010/2015 — already implemented as `SyntheticControl` — and is recapped only as the paper frames it. Nothing here is sourced from outside this paper.
@@ -107,7 +107,7 @@ reject the sharp null Eq. 11 if `p^f(φ, v) < γ`. The exact null (Eq. 6) is the
107107
(17) H_0^c̃: Y^I_{j,t} = Y^N_{j,t} + c̃ · (t − T0) · 𝟙[t ≥ T0+1] (linear-in-time, zero intercept)
108108
(18) C̃S_{(1−γ)}(φ, v) := { f : f = c̃·(t−T0)·𝟙[t ≥ T0+1] and p^c̃(φ) > γ } (confidence SET)
109109

110-
Operationally: grid over the scalar `c` (or ``), test each value via Eqs. 12–13, and collect the values that are not rejected. Extending to two-parameter functions (quadratic / exponential / logarithmic) is "theoretically straightforward" from Eq. 14 but computationally heavier; the paper restricts its main examples to one parameter. Confidence sets summarize **significance** (is `f ≡ 0` excluded?), **precision** (narrower ⇒ stronger conclusions), and **robustness** (compare set areas across `φ`). They are **uniform** over time (they combine information across all post periods to describe which effect *functions* are not rejected); a **point-wise** per-period CI instead uses `α̂_{1,t'}` as the test statistic separately for each `t' > T0` (Section 6.1 cautions that a point-wise interval may be inadequate).
110+
Operationally: grid over the scalar `c` (or `c̃`), test each value via Eqs. 12–13, and keep those satisfying the set's defining **strict** inequality `p^c(φ) > γ` (Eqs. 14/16/18). The strict `>` matters: a value with `p^f = γ` is *excluded* from the confidence set even though the rejection rule (`reject if p^f < γ`) does not reject it — the CI/CS boundary (`> γ`) and the test's acceptance region (`≥ γ`) differ exactly at `p^f = γ`, which is reachable because the permutation p-value is discrete (a multiple of `1/(J+1)`). Extending to two-parameter functions (quadratic / exponential / logarithmic) is "theoretically straightforward" from Eq. 14 but computationally heavier; the paper restricts its main examples to one parameter. Confidence sets summarize **significance** (is `f ≡ 0` excluded?), **precision** (narrower ⇒ stronger conclusions), and **robustness** (compare set areas across `φ`). They are **uniform** over time (they combine information across all post periods to describe which effect *functions* are not rejected); a **point-wise** per-period CI instead uses `α̂_{1,t'}` as the test statistic separately for each `t' > T0` (Section 6.1 cautions that a point-wise interval may be inadequate).
111111

112112
### Other test statistics + Monte Carlo (Section 5)
113113

0 commit comments

Comments
 (0)