the “original CI” much wider than DID CI

[YutingYale]

Thank you very much for developing and maintaining this package - it has been very helpful.

I am applying HonestDiD to a Callaway–Sant’Anna dynamic event-study (with bstrap = TRUE, min_e = -4, max_e = 6). The DID event-study shows significant post-treatment effects (aggregated) with reasonably tight CIs. However, the “original CI” from HonestDiD (at M = 0) is much wider and becomes non-significant.

It seems to me that the discrepancy is because the HonestDiD by default performs the sensitivity analysis for the first period. Is there an option I can specify to perform a sensitivity analysis of the aggregated effect? I saw the example of using c(0.5, 0.5) for the average of two periods, but I'm not sure if this applies to the case of the CS estimator and how it accounts for the weights across the periods.

Thank you very much for your help.

[jonathandroth]

Hello -- the original CI should match the CI you're getting from Callaway and Sant'Anna, so this is suggestive of some kind of import issue. The l_vec parameter determines the estimand that HonestDiD looks at, and by default is set to look at the first period in the event-study. If you think there is a bug beyond this, please feel free to provide a minimal reproducible example. J

…

On Sat, Dec 13, 2025 at 6:20 PM YutingYale ***@***.***> wrote: *YutingYale* created an issue (asheshrambachan/HonestDiD#67) <#67> Thank you very much for developing and maintaining this package - it has been very helpful. I am applying HonestDiD to a Callaway–Sant’Anna dynamic event-study (with bstrap = TRUE, min_e = -4, max_e = 6). The DID event-study shows significant post-treatment effects (aggregated) with reasonably tight CIs. However, the “original CI” from HonestDiD (at M = 0) is much wider and becomes non-significant. Is it expected that the HonestDiD original CI differs substantially from the standard DID CI in this setting? This happened for a few ouctomes but not all ouctomes, which is confusing to me. Thank you very much for your help. — Reply to this email directly, view it on GitHub <#67>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFC526PBZ6A54SXQKLD4BSNLRAVCNFSM6AAAAACO6WXAJOVHI2DSMVQWIX3LMV43ASLTON2WKOZTG4ZDMNBZGQ4TMNQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[YutingYale]

Hello -- the original CI should match the CI you're getting from Callaway
and Sant'Anna, so this is suggestive of some kind of import issue. The
l_vec parameter determines the estimand that HonestDiD looks at, and by
default is set to look at the first period in the event-study. If you think
there is a bug beyond this, please feel free to provide a minimal
reproducible example.

J
…

Thank you so much for the clarification. I just wanted to confirm my understanding: the “original” CI in HonestDiD should correspond to the CI for the estimand it uses - so if the default reference is the first post-treatment period, the original CI should match that period’s CI, correct?

In the GitHub example you provided, the CIs are very close but not identical (the Honest original CI seems a bit narrow). For example, at 𝑒=0, the CS CI is (0.0291, 0.0614), but the original CI from HonestDiD has an upper bound slightly below 0.06. I wanted to check whether this small difference is expected based on how HonestDiD constructs its baseline CI?

[jonathandroth]

HonestDiD forms the CI by taking a grid of points (governed by num.gridpoints) so there may be some small numerical differences

…

On Mon, Dec 15, 2025 at 10:36 AM YutingYale ***@***.***> wrote: *YutingYale* left a comment (asheshrambachan/HonestDiD#67) <#67 (comment)> Hello -- the original CI should match the CI you're getting from Callaway and Sant'Anna, so this is suggestive of some kind of import issue. The l_vec parameter determines the estimand that HonestDiD looks at, and by default is set to look at the first period in the event-study. If you think there is a bug beyond this, please feel free to provide a minimal reproducible example. J … <#m_-8055178591164784939_> Thank you so much for the clarification. I just wanted to confirm my understanding: the “original” CI in HonestDiD should correspond to the CI for the estimand it uses - so if the default reference is the first post-treatment period, the original CI should match that period’s CI, correct? In the GitHub example you provided, the CIs are very close but not identical (the Honest original CI seems a bit narrow). For example, at 𝑒=0, the CS CI is (0.0291, 0.0614), but the original CI from HonestDiD has an upper bound slightly below 0.06. I wanted to check whether this small difference is expected based on how HonestDiD constructs its baseline CI? — Reply to this email directly, view it on GitHub <#67 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFC4WJHMBWO6BLCHBS34B3IN5AVCNFSM6AAAAACO6WXAJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMNJWGI4TQMBUHA> . You are receiving this because you commented.Message ID: ***@***.***>