Running LightningCLI from Python cannot pass a config.yaml file #20722

2025-04-16 17:54:57,872 - LightningArgumentParser - DEBUG - Loaded parser defaults: Namespace(config=None, seed_everything=True, trainer=Namespace(accelerator='auto', strategy='auto', devices='auto', num_nodes=1, precision=None, logger=None, callbacks=None, fast_dev_run=False, max_epochs=None, min_epochs=None, max_steps=-1, min_steps=None, max_time=None, limit_train_batches=None, limit_val_batches=None, limit_test_batches=None, limit_predict_batches=None, overfit_batches=0.0, val_check_interval=None, check_val_every_n_epoch=1, num_sanity_val_steps=None, log_every_n_steps=None, enable_checkpointing=None, enable_progress_bar=None, enable_model_summary=None, accumulate_grad_batches=1, gradient_clip_val=None, gradient_clip_algorithm=None, deterministic=None, benchmark=None, inference_mode=True, use_distributed_sampler=True, profiler=None, detect_anomaly=False, barebones=False, plugins=None, sync_batchnorm=False, reload_dataloaders_every_n_epochs=0, default_root_dir=None), model=Namespace(out_dim=10), optimizer=None, lr_scheduler=None)
2025-04-16 17:54:57,878 - LightningArgumentParser - DEBUG - Skipping parameter "params" from "torch.optim.Adam.__init__" because of: Parameter requested to be skipped.
2025-04-16 17:54:57,880 - LightningArgumentParser - DEBUG - Parsed object: Namespace()
2025-04-16 17:54:57,897 - LightningArgumentParser - DEBUG - Skipping parameter "optimizer" from "torch.optim.lr_scheduler.CosineAnnealingLR.__init__" because of: Parameter requested to be skipped.
2025-04-16 17:54:57,898 - LightningArgumentParser - ERROR - Parser key "T_max":
  Expected a <class 'int'>. Got value: ${trainer.max_epochs}
2025-04-16 17:54:57,898 - LightningArgumentParser - ERROR - Parser key "lr_scheduler":
  Does not validate against any of the Union subtypes
  Subtypes: [<class 'NoneType'>, <class 'torch.optim.lr_scheduler.LRScheduler'>, <class 'lightning.pytorch.cli.ReduceLROnPlateau'>]
  Errors:
    - Expected a <class 'NoneType'>
    - Problem with given class_path 'torch.optim.lr_scheduler.CosineAnnealingLR':
        Parser key "T_max":
          Expected a <class 'int'>. Got value: ${trainer.max_epochs}
    - Import path torch.optim.lr_scheduler.CosineAnnealingLR does not correspond to a subclass of ReduceLROnPlateau
  Given value type: <class 'dict'>
  Given value: {'class_path': 'torch.optim.lr_scheduler.CosineAnnealingLR', 'init_args': {'T_max': '${trainer.max_epochs}'}}
2025-04-16 17:54:57,898 - LightningArgumentParser - DEBUG - Debug enabled, thus raising exception instead of exit.
Traceback (most recent call last):
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_typehints.py", line 838, in adapt_typehints
    vals.append(adapt_typehints(val, subtype, **adapt_kwargs))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_typehints.py", line 787, in adapt_typehints
    raise_unexpected_value(f"Expected a {typehint}", val)
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_typehints.py", line 718, in raise_unexpected_value
    raise ValueError(message) from exception
ValueError: Expected a <class 'NoneType'>. Got value: {'class_path': 'torch.optim.lr_scheduler.CosineAnnealingLR', 'init_args': {'T_max': '${trainer.max_epochs}'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_typehints.py", line 597, in _check_type
    raise ex
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_typehints.py", line 582, in _check_type
    val = adapt_typehints(val, self._typehint, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_typehints.py", line 846, in adapt_typehints
    raise_union_unexpected_value(sorted_subtypes, val, vals)
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_typehints.py", line 725, in raise_union_unexpected_value
    raise ValueError(
ValueError: Does not validate against any of the Union subtypes
Subtypes: [<class 'NoneType'>, <class 'torch.optim.lr_scheduler.LRScheduler'>, <class 'lightning.pytorch.cli.ReduceLROnPlateau'>]
Errors:
  - Expected a <class 'NoneType'>
  - Problem with given class_path 'torch.optim.lr_scheduler.CosineAnnealingLR':
      Parser key "T_max":
        Expected a <class 'int'>. Got value: ${trainer.max_epochs}
  - Import path torch.optim.lr_scheduler.CosineAnnealingLR does not correspond to a subclass of ReduceLROnPlateau
Given value type: <class 'dict'>
Given value: {'class_path': 'torch.optim.lr_scheduler.CosineAnnealingLR', 'init_args': {'T_max': '${trainer.max_epochs}'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_core.py", line 505, in parse_object
    cfg_apply = self._apply_actions(cfg_obj, prev_cfg=cfg)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_core.py", line 1372, in _apply_actions
    value = self._check_value_key(action, value, action_dest, prev_cfg)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_core.py", line 1424, in _check_value_key
    value = action._check_type_(value, cfg=cfg)  # type: ignore[attr-defined]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_common.py", line 333, in _check_type_
    return self._check_type(value, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_typehints.py", line 610, in _check_type
    raise TypeError(f'Parser key "{self.dest}"{elem}:\n{error}') from ex
TypeError: Parser key "lr_scheduler":
  Does not validate against any of the Union subtypes
  Subtypes: [<class 'NoneType'>, <class 'torch.optim.lr_scheduler.LRScheduler'>, <class 'lightning.pytorch.cli.ReduceLROnPlateau'>]
  Errors:
    - Expected a <class 'NoneType'>
    - Problem with given class_path 'torch.optim.lr_scheduler.CosineAnnealingLR':
        Parser key "T_max":
          Expected a <class 'int'>. Got value: ${trainer.max_epochs}
    - Import path torch.optim.lr_scheduler.CosineAnnealingLR does not correspond to a subclass of ReduceLROnPlateau
  Given value type: <class 'dict'>
  Given value: {'class_path': 'torch.optim.lr_scheduler.CosineAnnealingLR', 'init_args': {'T_max': '${trainer.max_epochs}'}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\my_project\run_multiple_seeds.py", line 18, in <module>
    cli = cli_main(args=config)
          ^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\main.py", line 27, in cli_main
    cli = MyLightningCLI(
          ^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\lightning\pytorch\cli.py", line 383, in __init__
    self.parse_arguments(self.parser, args)
  File "C:\my_project\.venv\Lib\site-packages\lightning\pytorch\cli.py", line 532, in parse_arguments
    self.config = parser.parse_object(args)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_deprecated.py", line 123, in patched_parse
    cfg = parse_method(*args, _skip_validation=_skip_validation, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_core.py", line 518, in parse_object
    self.error(str(ex), ex)
  File "C:\my_project\.venv\Lib\site-packages\jsonargparse\_core.py", line 1065, in error
    raise argument_error(message) from ex
argparse.ArgumentError: Parser key "lr_scheduler":
  Does not validate against any of the Union subtypes
  Subtypes: [<class 'NoneType'>, <class 'torch.optim.lr_scheduler.LRScheduler'>, <class 'lightning.pytorch.cli.ReduceLROnPlateau'>]
  Errors:
    - Expected a <class 'NoneType'>
    - Problem with given class_path 'torch.optim.lr_scheduler.CosineAnnealingLR':
        Parser key "T_max":
          Expected a <class 'int'>. Got value: ${trainer.max_epochs}
    - Import path torch.optim.lr_scheduler.CosineAnnealingLR does not correspond to a subclass of ReduceLROnPlateau
  Given value type: <class 'dict'>
  Given value: {'class_path': 'torch.optim.lr_scheduler.CosineAnnealingLR', 'init_args': {'T_max': '${trainer.max_epochs}'}}

2025-04-16 18:01:33,959 - LightningArgumentParser - DEBUG - Loaded parser defaults: Namespace(config=None, seed_everything=True, trainer=Namespace(accelerator='auto', strategy='auto', devices='auto', num_nodes=1, precision=None, logger=None, callbacks=None, fast_dev_run=False, max_epochs=None, min_epochs=None, max_steps=-1, min_steps=None, max_time=None, limit_train_batches=None, limit_val_batches=None, limit_test_batches=None, limit_predict_batches=None, overfit_batches=0.0, val_check_interval=None, check_val_every_n_epoch=1, num_sanity_val_steps=None, log_every_n_steps=None, enable_checkpointing=None, enable_progress_bar=None, enable_model_summary=None, accumulate_grad_batches=1, gradient_clip_val=None, gradient_clip_algorithm=None, deterministic=None, benchmark=None, inference_mode=True, use_distributed_sampler=True, profiler=None, detect_anomaly=False, barebones=False, plugins=None, sync_batchnorm=False, reload_dataloaders_every_n_epochs=0, default_root_dir=None), model=Namespace(out_dim=10), optimizer=None, lr_scheduler=None)
2025-04-16 18:01:33,965 - LightningArgumentParser - DEBUG - Parsed object: {'config': 'config.yaml', 'seed_everything': 42}
Seed set to 42
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
C:\my_project\.venv\Lib\site-packages\lightning\pytorch\loops\utilities.py:72: `max_epochs` was not set. Setting it to 1000 epochs. To train without an epoch limit, set `max_epochs=-1`.
Traceback (most recent call last):
  File "C:\my_project\run_multiple_seeds.py", line 23, in <module>
    cli.trainer.fit(cli.model, cli.datamodule)
  File "C:\my_project\.venv\Lib\site-packages\lightning\pytorch\trainer\trainer.py", line 538, in fit
    call._call_and_handle_interrupt(
  File "C:\my_project\.venv\Lib\site-packages\lightning\pytorch\trainer\call.py", line 47, in _call_and_handle_interrupt
    return trainer_fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\my_project\.venv\Lib\site-packages\lightning\pytorch\trainer\trainer.py", line 574, in _fit_impl
    self._run(model, ckpt_path=ckpt_path)
  File "C:\my_project\.venv\Lib\site-packages\lightning\pytorch\trainer\trainer.py", line 931, in _run
    _verify_loop_configurations(self)
  File "C:\my_project\.venv\Lib\site-packages\lightning\pytorch\trainer\configuration_validator.py", line 36, in _verify_loop_configurations
    __verify_train_val_loop_configuration(trainer, model)
  File "C:\my_project\.venv\Lib\site-packages\lightning\pytorch\trainer\configuration_validator.py", line 59, in __verify_train_val_loop_configuration
    raise MisconfigurationException(
lightning.fabric.utilities.exceptions.MisconfigurationException: No `configure_optimizers()` method defined. Lightning `Trainer` expects as minimum a `training_step()`, `train_dataloader()` and `configure_optimizers()` to be defined.