Skip to content

"Decompose Weights (DoRA)" broken for LoKr? #1602

Description

@dxqb

Discussed in #1601

@Koratahiu

Originally posted by gith82387234 July 8, 2026
From what I've read and my limited understanding of it, this option in the LoRa tab can greatly help with overfitting, especially when replacing concepts... However enabling this option throws an error which disappears when disabled. Is it just not compatible with LoKr or could it be broken since LoKr support is relatively new?

The error in question:

Traceback (most recent call last):
  File "A:\OneTrainer\modules\ui\TrainUIController.py", line 226, in __training_thread_function
    trainer.start()
  File "A:\OneTrainer\modules\trainer\GenericTrainer.py", line 141, in start
    self.model_setup.setup_model(self.model, self.config)
  File "A:\Trainers\OneTrainer\modules\modelSetup\Krea2LoRASetup.py", line 62, in setup_model
    model.transformer_lora = LoRAModuleWrapper(
                             ^^^^^^^^^^^^^^^^^^
  File "A:\Trainers\OneTrainer\modules\module\LoRAModule.py", line 916, in __init__
    self.lora_modules = self.__create_modules(orig_module, config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "A:\Trainers\OneTrainer\modules\module\LoRAModule.py", line 961, in __create_modules
    lora_module = self.klass(prefixed_name, child_module, *self.additional_args, **self.additional_kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "A:\OneTrainer\modules\module\LoRAModule.py", line 328, in __init__
    self.initialize_weights()
  File "A:\OneTrainer\modules\module\LoRAModule.py", line 439, in initialize_weights
    self.dora_scale = Parameter(
                      ^^^^^^^^^^
  File "A:\OneTrainer\venv\Lib\site-packages\torch\nn\parameter.py", line 57, in __new__
    return torch.Tensor._make_subclass(cls, data, requires_grad)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

RuntimeError: Only Tensors of floating point and complex dtype can require gradients
```</div>

 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions