`smooth_estimates()` sometimes disagrees with `predict()` when `discrete = TRUE`

[dlmdata]

Hi Gavin!

I have noticed that in some cases smooth_estimates() can disagree with predict(m, type = "terms") when fitting models with bam using discrete = TRUE.

library("tidyverse")
library("mgcv")
library("gratia")

m1 <- bam(price ~ s(color, clarity, bs = "re"), data = diamonds, discrete = FALSE)
bind_cols(smooth_estimates(m1), predict(m1, smooth_estimates(m1), type = "terms"))

m2 <- bam(price ~ s(color, clarity, bs = "re"), data = diamonds, discrete = TRUE)
bind_cols(smooth_estimates(m2), predict(m2, smooth_estimates(m2), type = "terms"))

In the first model, .estimate is identical to s(color,clarity), but in the second model it is not.

[gavinsimpson]

Thanks for this; it's definitely the discretization; setting discrete = FALSE in the predict() call for the second example makes it give the same answers as smooth_estimates(). I will need to look at this; it might be that I need to do something special for bam(..., discrete = TRUE) models, but for that I would need to repeat the discretization and I'm not sure that is available from mgcv.

[gavinsimpson]

It appears that there is some reordering / remapping of the data that is not performed when discrete = FALSE is used in predict() (or in smooth_estimates()).

[gavinsimpson]

One solution seems to be to go via predict(...., type = "lpmatrix") and then post multiply by the coefs. Or via "terms" or "iterms"; should see which is the most computationally efficient. Right now, the PredictMat() approach I am using for gam() models doesn't work well with the discretization of bam() models.