Skip to content

Add MLFlowLogger.log_model to log and register models with MLflow - #22051

Open
Michael-Duck wants to merge 2 commits into
Lightning-AI:masterfrom
Michael-Duck:michael-duck/ai-21437-mlflow-log-model
Open

Michael-Duck wants to merge 2 commits into
Lightning-AI:masterfrom
Michael-Duck:michael-duck/ai-21437-mlflow-log-model

Conversation

@Michael-Duck

Copy link
Copy Markdown

Summary

Adds MLFlowLogger.log_model(), which logs a model to the MLflow run owned by the logger and optionally registers it in the MLflow model registry.

MLFlowLogger could only log checkpoints as plain MLflow artifacts, so the "Register Model" action of the MLflow UI was never available for models trained with Lightning. This adds the missing mlflow.log_model equivalent:

from lightning.pytorch import Trainer
from lightning.pytorch.loggers import MLFlowLogger

logger = MLFlowLogger(experiment_name="my-exp")
trainer = Trainer(logger=logger)
trainer.fit(model)

# log the model to the run and register it in the model registry
logger.log_model(model, registered_model_name="my-model")

The model is logged with MLflow's pytorch flavor, so it shows up as a model of the run in the MLflow UI and can be loaded again with mlflow.pyfunc.load_model("models:/my-model/1").

Notes

  • The model is logged into the run that the logger manages, via mlflow.models.Model.log(..., run_id=...). mlflow.pytorch.log_model() would log into the globally active MLflow run and terminate that run when it exits, which would interfere with the run owned by the logger.
  • serialization_format="pickle" is used by default when the installed MLflow supports it, because the newer pt2 format needs an input example to trace the model, which the logger cannot infer. Pass input_example=... (or serialization_format=...) to override this.
  • The log_model argument of the constructor is untouched and keeps controlling whether the checkpoints are logged as artifacts. The new method logs a model object instead of a checkpoint.

Tests

pytest tests/tests_pytorch/loggers/test_mlflow.py

  • new test test_mlflow_logger_log_model: without this change it fails with AttributeError: 'MLFlowLogger' object has no attribute 'log_model', with this change it passes (it logs the model into the run, registers it in the model registry and loads it back for inference).
  • whole file with this change: 21 passed, 1 failed, 2 errors. The failure (test_mlflow_run_name_setting) and the two errors (the environment variable leak check tripping over MLFLOW_TRACKING_URI, which MLflow 3.x exports) are pre-existing on this machine with MLflow 3.17 and fail identically without this change (20 passed, 1 failed, 2 errors).

pytest tests/tests_pytorch/loggers/ (wider run): 107 passed, 32 failed, 2 skipped, 6 xfailed, 2 errors. All failures are environmental and unrelated to this change: tensorboard/tensorboardX and jsonargparse are not installed, and the ddp_spawn based test_logger_initialization cases cannot spawn processes in this sandbox (PermissionError: [Errno 1] Operation not permitted).

ruff check and ruff format --check on the changed files pass, sphinx-lint passes on the changed documentation.

Fixes #21437

…ightning-AI#21437)

feat: add `MLFlowLogger.log_model` to log a model to the MLflow run and to
optionally register it in the MLflow model registry

`MLFlowLogger` could only log checkpoints as plain MLflow artifacts, so the
"Register Model" action of the MLflow UI was never available for models
trained with Lightning. This adds the missing `mlflow.log_model` equivalent:

```python
logger = MLFlowLogger(experiment_name="my-exp")
trainer = Trainer(logger=logger)
trainer.fit(model)
logger.log_model(model, registered_model_name="my-model")
```

The model gets logged with the `pytorch` flavor of MLflow into the run owned
by the logger and, if `registered_model_name` is given, registered in the
MLflow model registry.

Fixes Lightning-AI#21437

Signed-off-by: michael-duck <michael@realms.nexus>
…nagers guide

Signed-off-by: michael-duck <michael@realms.nexus>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MLflow model format required for the Register Model

1 participant