Trending CIs from smoothness restriction

[jacob-smith-econ]

Unsure if this is a code issue or an issue with my application, but these trending CIs seem rather odd. I've looked around and have not been able to find any references on this.

I did get the following warning:

"Solution may be inaccurate. Try another solver, adjusting solver settings, or solve with `verbose=TRUE` for more information."

Any commentary would be great.

[jonroth]

Hi - thanks for the message. The jump from the red (original) to blue is not surprising if the pre-trend is non-zero, since M=0 assumes linear trends and so should be centered around a (weighted) linear extrapolation of the pre-trends rather than the assumption of exact parallel trends. The weird part about the graph is that the FLCIs are non-nested as you increase M. I have seen this happen in cases where the pre-trend is in fact non-linear and you impose a small value of M which contradicts the non-linearity of the pre-trends; the reason is that the FLCI for different values of M puts different weights on different pre-periods. Do the FLCIs become nested if you increase the value of M? If so, that is likely the reason. Although I am not familiar with that warning you presented, so it could also be that something is off with the numerical optimization. Sorry for not having a more definitive answer.

J

[jacob-smith-econ]

Increasing step size seems to make it weirder but it seems to stabilize.

For references I have included my event study plot as well. The coefficient of interest is 2017.

When analyzing the 2016 coefficient this seems better behaved.

Choice of grid seems to matter here.

[jonroth]

yes, everything but the first bin looks fine in that zoomed out plot. my guess is the weirdness of the first bin is coming from the fact that you're imposing deviations from linarity of no more than ~0.04, but from the graph it looks like the slope between the two preperiods changes more than that, so in some sense you're imposing a linearity restriction that's violated by the data.

[jacob-smith-econ]

Interesting!

Are there any restrictions under the Honest DiD Framework which precludes this behavior? Maybe some sort of monotonicity restriction on the upper and lower CI.

Maybe you discuss this in more depth in the paper, but some guidelines on grid size which precludes weird action is a good idea.

Just some food for thought.

Thanks for helping me understand this!

[jonathandroth]

Thanks for the comments! - It's been a while since i've thought about this, but we do have a function HonestDiD:::DeltaSD_upperBound_Mpre which returns a value M_{pre}^{ub} which is guaranteed to be larger than the maximal pre-treatment value of M with probability 1-alpha. Values of M larger than that are clearly consistent with the data and thus unlikely to exhibit weird behavior in terms of the FLCIs. (Setting alpha=0.5 instead of 0.05 gives you a point estimate of the observed pre-treatment non-linearity instead of a CI). In principle you can also ensure the CIs are nested by using the same affine estimator for all values of M. Currently, we optimize the affine estimator for each value of M to give the shortest CI. Unfortunately I'm not sure we have the RA resources to implement this as an option right now (it has come up rarely, so you're unlucky I guess :)).

…

On Thu, Aug 13, 2026 at 10:08 AM Jacob Smith ***@***.***> wrote: *jacob-smith-econ* left a comment (asheshrambachan/HonestDiD#71) <#71 (comment)> Interesting! Are there any restrictions under the Honest DiD Framework which precludes this behavior? Maybe some sort of monotonicity restriction on the upper and lower CI. Maybe you discuss this in more depth in the paper, but some guidelines on grid size which precludes weird action is a good idea. Just some food for thought. Thanks for helping me understand this! — Reply to this email directly, view it on GitHub <#71?email_source=notifications&email_token=AE6EXFELOBQO5UHGPR3BPW35JXDT5A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMRYGE2TEMRSG43KM4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#issuecomment-5281522276>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFFMI34PRCDGCDJMDO35JXDT5AVCNFSNUABFKJSXA33TNF2G64TZHMZDEMBTGUYDSNBUHNEXG43VMU5TKMJTGU2TGMRZGQ4KC5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AE6EXFB5UR2MN76CNNBSSN35JXDT5A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMRYGE2TEMRSG43KM4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSVGM33PORSXEX3JN5ZQ> and Android <https://github.com/notifications/mobile/android/AE6EXFFXOLZFXHQNZDOG2AL5JXDT5A5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMRYGE2TEMRSG43KM4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSXGM33PORSXEX3BNZSHE33JMQ>. Download it today! You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jacob-smith-econ]

Maybe some general guidance on the wiki for these cases. In particular what it means for there to be

1. Problem1a: Trending CIs- Fix: Expand grid size to see how big the issue is.

2. Problem1b: Non-nested Cis Solution: HonestDiD:::DeltaSD_upperBound_Mpre (?)

Note that if im really "Dishonest" with the smoothness restriction CIs, I can just move in increments of ~0.05 to hide this jump. It could be that im making a mountain out of a molehill, still it seems strange that CIs would move this way.

[jonroth]

Thanks for the suggestions!

BTW, would you be willing to share the beta and sigma you used here?

Although I told you what could be generating the non-monotonic CIs here, I am a bit concerned by the warning you reported of "Solution may be inaccurate. Try another solver, adjusting solver settings, or solve with verbose=TRUE for more information." It would be useful to have someone trace this down to make sure this non-monotonicity is real and not a numerical issue with the solver.

[jacob-smith-econ]

Sure. I have attached them here.

model_results.xlsx