@kmehant The update for subclassing the model change creates lots of missing module warnings, specifically when loading a trained model with .from_pretrained.
like this, for ibm-granite/granite-4.0-tiny-preview. I seem to remember seeing similar warnings for other models when I initially made this package, which is why I didn't subclass.
It does seem that these warnings aren't actually resulting in bad outputs, though, so not sure what the warning logic actually is
UserWarning: Found missing adapter keys while loading the checkpoint: ['base_model.model.model.layers.0.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.0.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.0.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.0.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.1.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.1.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.1.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.1.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.2.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.2.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.2.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.2.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.3.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.3.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.3.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.3.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.4.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.4.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.4.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.4.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.5.self_attn.q_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.5.self_attn.q_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.5.self_attn.k_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.5.self_attn.k_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.5.self_attn.v_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.5.self_attn.v_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.5.self_attn.o_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.5.self_attn.o_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.6.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.6.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.6.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.6.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.7.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.7.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.7.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.7.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.8.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.8.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.8.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.8.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.9.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.9.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.9.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.9.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.10.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.10.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.10.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.10.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.11.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.11.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.11.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.11.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.12.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.12.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.12.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.12.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.13.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.13.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.13.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.13.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.14.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.14.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.14.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.14.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.15.self_attn.q_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.15.self_attn.q_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.15.self_attn.k_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.15.self_attn.k_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.15.self_attn.v_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.15.self_attn.v_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.15.self_attn.o_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.15.self_attn.o_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.16.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.16.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.16.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.16.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.17.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.17.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.17.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.17.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.18.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.18.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.18.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.18.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.19.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.19.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.19.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.19.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.20.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.20.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.20.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.20.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.21.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.21.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.21.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.21.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.22.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.22.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.22.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.22.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.23.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.23.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.23.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.23.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.24.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.24.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.24.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.24.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.25.self_attn.q_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.25.self_attn.q_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.25.self_attn.k_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.25.self_attn.k_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.25.self_attn.v_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.25.self_attn.v_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.25.self_attn.o_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.25.self_attn.o_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.26.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.26.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.26.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.26.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.27.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.27.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.27.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.27.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.28.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.28.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.28.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.28.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.29.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.29.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.29.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.29.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.30.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.30.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.30.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.30.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.31.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.31.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.31.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.31.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.32.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.32.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.32.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.32.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.33.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.33.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.33.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.33.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.34.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.34.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.34.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.34.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.35.self_attn.q_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.35.self_attn.q_proj.base_layer.lora_B.constraint.weight',
@kmehant The update for subclassing the model change creates lots of missing module warnings, specifically when loading a trained model with .from_pretrained.
like this, for ibm-granite/granite-4.0-tiny-preview. I seem to remember seeing similar warnings for other models when I initially made this package, which is why I didn't subclass.
It does seem that these warnings aren't actually resulting in bad outputs, though, so not sure what the warning logic actually is
UserWarning: Found missing adapter keys while loading the checkpoint: ['base_model.model.model.layers.0.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.0.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.0.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.0.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.1.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.1.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.1.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.1.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.2.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.2.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.2.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.2.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.3.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.3.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.3.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.3.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.4.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.4.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.4.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.4.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.5.self_attn.q_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.5.self_attn.q_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.5.self_attn.k_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.5.self_attn.k_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.5.self_attn.v_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.5.self_attn.v_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.5.self_attn.o_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.5.self_attn.o_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.6.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.6.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.6.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.6.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.7.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.7.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.7.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.7.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.8.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.8.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.8.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.8.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.9.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.9.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.9.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.9.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.10.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.10.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.10.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.10.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.11.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.11.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.11.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.11.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.12.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.12.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.12.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.12.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.13.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.13.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.13.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.13.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.14.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.14.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.14.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.14.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.15.self_attn.q_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.15.self_attn.q_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.15.self_attn.k_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.15.self_attn.k_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.15.self_attn.v_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.15.self_attn.v_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.15.self_attn.o_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.15.self_attn.o_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.16.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.16.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.16.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.16.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.17.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.17.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.17.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.17.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.18.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.18.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.18.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.18.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.19.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.19.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.19.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.19.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.20.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.20.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.20.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.20.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.21.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.21.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.21.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.21.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.22.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.22.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.22.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.22.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.23.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.23.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.23.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.23.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.24.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.24.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.24.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.24.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.25.self_attn.q_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.25.self_attn.q_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.25.self_attn.k_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.25.self_attn.k_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.25.self_attn.v_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.25.self_attn.v_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.25.self_attn.o_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.25.self_attn.o_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.26.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.26.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.26.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.26.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.27.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.27.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.27.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.27.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.28.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.28.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.28.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.28.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.29.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.29.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.29.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.29.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.30.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.30.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.30.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.30.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.31.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.31.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.31.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.31.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.32.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.32.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.32.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.32.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.33.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.33.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.33.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.33.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.34.mamba.in_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.34.mamba.in_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.34.mamba.out_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.34.mamba.out_proj.base_layer.lora_B.constraint.weight', 'base_model.model.model.layers.35.self_attn.q_proj.base_layer.lora_A.constraint.weight', 'base_model.model.model.layers.35.self_attn.q_proj.base_layer.lora_B.constraint.weight',