Exporting experiment on Databricks

[tahaum]

Hi! I'm trying to export an experiment to a folder using Databricks. I've got it to work running the commands listed in the README locally, but when trying the same in my Databricks workspace I'm running into trouble. After building the wheel, pushing it to DBFS and installing it on the cluster I'm trying the following:

%sh

python -u -m mlflow_export_import.experiment.export_experiment \
  --experiment /Shared/my_experiment \
  --output-dir /dbfs/mnt/out \
  --notebook-formats SOURCE

I then get the following error:

Traceback (most recent call last):
File "/databricks/conda/envs/databricks-ml/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/databricks/conda/envs/databricks-ml/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/mlflow_export_import/experiment/export_experiment.py", line 82, in
main()
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/mlflow_export_import/experiment/export_experiment.py", line 78, in main
exporter = ExperimentExporter(None, export_metadata_tags, utils.string_to_list(notebook_formats))
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/mlflow_export_import/experiment/export_experiment.py", line 22, in init
self.fs = filesystem or _filesystem.get_filesystem()
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/mlflow_export_import/common/filesystem.py", line 50, in get_filesystem
return DatabricksFileSystem() if use_databricks else LocalFileSystem()
File "/databricks/conda/envs/databricks-ml/lib/python3.8/site-packages/mlflow_export_import/common/filesystem.py", line 13, in init
self.dbutils = IPython.get_ipython().user_ns["dbutils"]
AttributeError: 'NoneType' object has no attribute 'user_ns'

Is this the right approach when running export/import on Databricks?

[dharmeshpurohit]

I am also getting same issue is there any resolution to this issue?

[amesar]

Try using these Databricks notebooks: https://github.com/amesar/mlflow-export-import#databricks-notebooks