why MEMIT+'s adj_k add layer_kp @ layer_kp.T?

[jasonrichdarmawan]

Hi,

I am trying to understand AnyEdit. However, I am not sure why we add layer_kp @ layer_kp.T when computing adj_k. I understand MEMIT weight update is from the objective function, which does not have layer_kp @ layer_kp.T, but what about MEMIT+'s?

Note: my motivation is to understand how the objective function derived into weight update, like MEMIT. I am especially interested why layer_kp is only used in the left side of AX=B and there is comment of layer_kp @ layer_kp.T in the right side

AnyEdit/memit_ARE/memit_ARE_main.py

Lines 139 to 142 in ac8e652

	adj_k = torch.linalg.solve(
	hparams.mom2_update_weight * cov + layer_kp @ layer_kp.T +layer_ks @ layer_ks.T, #layer_kp @ layer_kp.T +
	layer_ks
	)