Different variance per stratum

[ArnoStrouwen]

The documentation states that all observations are subject to the same noise term σ²I.
Is it possible to give different groups their own variance?
For example, if my group is gender and the first two observations are M and the third is F, I would want:

σ²M 0 0
0 σ²M 0
0 0 σ²F

i.e. something similar to varIdent in R:
https://cran.r-project.org/web/packages/nlme/nlme.pdf#Rfn.varIdent

[palday]

This is not currently possible and it's not exactly trivial to implement in general.

I need to update the docs here, but I have some [lecture notes with comments on limitations of MixedModels.jl] and mention a possible alternative.

[ajinkya-k]

@palday would it be possible for you share those lecture notes? I also need this for a client project

[palday]

Oh, I meant to post the link in my original comment, but here it is now 😄 https://palday.github.io/economics2024/01-intro.html#limitations-of-mixedmodels.jl

[ArnoStrouwen]

That document states:

MixedModels.jl does support constraining the residual variance to known scalar value, which is useful in meta-analysis.

Can that value be zero? If so, I think I might be able to wrangle MixedModels enough to produce my model structure by having σ²M coming from a custom contrast coding, σ²F coming from a (different?) custom contrast coding, σ²=0.

I don't think I've come across that constraining options in the docs, maybe it is too well hidden.

[palday]

That document states:

MixedModels.jl does support constraining the residual variance to known scalar value, which is useful in meta-analysis.

Can that value be zero? If so, I think I might be able to wrangle MixedModels enough to produce my model structure by having σ²M coming from a custom contrast coding, σ²F coming from a (different?) custom contrast coding, σ²=0.

If I recall correctly, there are no checks on the value of sigma, but I don't know what will happen numerically. I suspect nothing good because everything is computed on the relative covariance scale, i.e. as multiple of the residual standard deviation, so if you force to be zero, then 0 * x = 0 for all real x and things will breakdown.

Instead, I suspect that the way to go is something like

fit(MixedModel, @formula(y ~ x + .... + (1|sex)), data; σ=1)

but I'm answering quickly between meetings and haven't thought it all the way through.

Do you have a small chunk of data + model you could share?

[palday]

Wait, is the actual desired constraint simply a diagonal covariance structure? Something like the || in lme4?

[ArnoStrouwen]

I don't think so, but I'm not an R expert.
I'll simulate some data to make it clear sometime next week.

[dmbates]

Wait, is the actual desired constraint simply a diagonal covariance structure? Something like the || in lme4?

What is desired here is modification to the covariance structure of the distribution of Y, not of B. Our computational methods are strongly dependent on the distribution of Y having a "sigma-squared I" covariance.

It is not to say that this assumption can't be modified; you could view the methods for GLMMs as doing something like this, but it would be comparable in complexity to going from LMMs to GLMMs.