How are sampling and covariate balance weights combined in glm_weightit() and avg_comparisons()?

[LauraCachon]

Hello,

I am following your example code https://ngreifer.github.io/WeightIt/articles/estimating-effects.html and https://ngreifer.github.io/blog/treatment-effects-mi/ to estimate ATEs after weighting with multiply imputed data, including attrition weights (using the sampling weights option to incorporate the attrition weights).

We are wondering whether this approach of combining attrition and covariate balance weights is defensible, and for this it would be very useful to better understand how weights are combined within glm_weight() and avg_comparisons(). Are both sets of weights getting multiplied internally?

Thank you so much in advance!

I'm attaching the code in case it helps to understand our doubt:

#Attrition weights
w_attr <- weightthem(
  selected ~ cm1age + race_merged + race1_mother + mother_edu3 + mborn + cognit3_mother + relst1,
  data = imp_mids, method = "ps")

#CB weights
w.gbm_att <- weightthem(lon_3l ~ sex_child1 + race_merged + lowbbweight + cm1age + race1_mother + mborn + relst1 + mother_edu3 + cognit3_mother
                        + cm3md_case_con + cm3alc_case + cm3drug_case + cm3gad_case + m_health3 + health3 + disab3 + mh.scale + nce.scale + chmalt.scale + ppvt.scale + mh_ppvt + nce_ppvt + chmalt_ppvt,
                        data = imp_org_attmids, approach = 'within', method = "gbm", 
                        estimand = "ATE", s.weights = "w_attr")

#Outcome model and comparisons
fits <- lapply(seq_along(w.gbm_att$models), function(i) {
  data <- complete(w.gbm_att, i)
  W <- w.gbm_att$models[[i]]
  
  glm_weightit(edu_nonacad ~ lon_3l + sex_child1 + race_merged + lowbbweight + cm1age + race1_mother + mborn + relst1 + mother_edu3 + cognit3_mother
               + cm3md_case_con + cm3alc_case + cm3drug_case + cm3gad_case + m_health3 + health3 + disab3 + mh.scale + nce.scale + chmalt.scale + mh_ppvt + nce_ppvt + chmalt_ppvt,
               data = data, 
               weightit = W, family = binomial, vcov = "HC0") # W because glm_weightit() automatically incorporates the sampling weights into the estimation of the outcome model
}) 
 
comp.imp <- lapply(fits, function(fit) {
  avg_comparisons(fit,
                  variables = list(lon_3l = "pairwise"),
                  comparison = "lnratioavg", wts = W$s.weights) })

pooled.try <- mice::pool(comp.imp, dfcom = Inf)
summary(pooled.try, conf.int = TRUE, exponentiate = T) 

[ngreifer]

Hi Laura,

Unfortunately, attrition weights are a bit different from sampling weights. You won't be able to use the existing guides or MatchThem functionality to correctly incorporate them. I should really write a guide on attrition weights, but as of now, WeightIt isn't really set up to accommodate them in a complete, official capacity. I'll describe conceptually how they should be implemented, and I'll try to provide some code to help you.

I'm assuming attrition takes place between treatment assignment and the outcome. Below are the steps you would need to run (regardless of software):

1. Estimate balancing weights in the full sample
2. Estimate attrition weights in the full sample
3. Fit the outcome model in the observed (non-dropout) sample using weights computed as the product of the balancing weights and the attrition weights
4. Perform unweighted g-computation in the full sample

With multiply imputed data, you would need to do this separately in each imputation.

You can either perform these steps separately as written, or you can "trick" the packages into doing what you want. I would recommend doing both methods and making sure you get identical results. First, I'll show you what it would look like to do each step separately as written:

# Balancing weights
bal_w <- weightit(tx ~ x1 + x2, data = data, estimand = "ATE")

# Selection weights
sel_w <- weightit(sel ~ tx + x1 + x2, data = data, estimand = "ATE")

# Multiply them together
data$out_w <- bal_w$weights * sel_w$weights

# Outcome model in the observed sample
sel_data <- subset(data, sel == 1)
fit <- glm(y ~ tx * (x1 + x2), data = sel_data, family = binomial, weights = out_w)

# G-comp in the full data
avg_comparisons(fit, variables = "tx", newdata = data, vcov = "HC3", comparison = "lnoravg")

This method treats both sets of weights as fixed. You would need to perform the whole analysis in each imputed dataset and pool the results at the end.

An alternative, slightly hacky method, is to do the following:

all_w <- weightitMSM(list(tx ~ x1 + x2, sel ~ tx + x1 + x2), data = data)

data$y[data$sel == 1] <- 0
data$off <- 0

fit <- glm_weightit(y ~ offset(off) + 0 + sel + sel:(tx * (x1 + x2)), data = data, family = binomial, weightit = all_w)

avg_comparisons(fit, variables = "tx", newdata = transform(data, sel = 1),  comparison = "lnoravg")

This method uses the full sample to fit both sets of weights and compute their product. Instead of only fitting the outcome model to the observed cases, we essentially fit two outcome models: one for the observed cases and one for the missing cases (the latter of which is just an intercept that we force to be 0). Finally, we perform g-computation on the full sample, setting the full sample to be observed (which means only the observed cases model is used for g-computation). Because the balancing and attrition weights are fit together using weightitMSM() and the outcome model is fit using glm_weightit(), the uncertainty in the weights can be properly account for. Again, you'd have to do this in each imputed dataset, and MatchThem won't help there. I haven't tested this method, so use it with caution.

[LauraCachon]

Hello Noah,

Thank you so much for your quick response. This is very informative and helpful!