Problematic multi-trial behavior in src/artifacts.jl:
function get_all_artifact_meta(study::Study)
return stack([get_all_artifact_meta(study, trial) for trial in study.study.trials])[1, :]
end
This function uses stack() to combine artifact metadata from all trials. The indexing [1, :] is problematic in several cases:
- If the study contains multiple trials without artifacts, or trials have differing numbers of artifacts, the stacking and slicing will throw an error (e.g., BoundsError or shape mismatch).
- This means
get_all_artifact_meta does not robustly handle studies with more than one trial, since the returned structure is not correctly shaped if trials differ in their artifact metadata.
Download artifact path issue:
function download_artifact(study::Study, artifact_id::String, file_path::String)
return optuna.artifacts.download_artifact(;
artifact_store=study.artifact_store.artifact_store,
artifact_id=artifact_id,
file_path=abspath(file_path) * "$artifact_id.jld2",
)
end
Here, line file_path=abspath(file_path) * "$artifact_id.jld2" means the file is always put within a directory, not at the exact path specified, possibly leading to confusion or mismatches when the path is treated as a file rather than a directory.
Test coverage does not exercise multiple-trial cases:
The test for get_all_artifact_meta in test/artifacts.jl only creates and tests a single trial:
create_test_study(; study_name="artifact_test") do study, test_dir
trial = ask(study)
...
metas = get_all_artifact_meta(study)
@test length(metas) == 1
end
This would not catch issues with multiple trials or errors thrown due to stacking in get_all_artifact_meta.
API usability issue for single-trial metadata lookup:
The function signature for get_all_artifact_meta(study::Study, trial) expects the second argument to be a Julia Trial object. However, internally this calls Python's optuna.artifacts.get_all_artifact_meta(trial; ...), which requires a Python trial object, i.e., trial.trial from a Julia Trial.
This can cause confusion and errors: calling get_all_artifact_meta(study, trial) with a Julia Trial works, but attempting to call it with a Python trial or a Trial.trial object (as needed by the Python API) will not work as expected.
Summary:
- Using
get_all_artifact_meta on a study with multiple trials will throw errors, limiting its utility for real-world use cases where studies have many trials.
- The handling of artifact paths may also cause confusion or unintended results when downloading artifacts.
- The interface for single-trial artifact metadata is inconsistent: the Julia wrapper accepts a Julia
Trial, but the call to the Python backend must get .trial, which is easy to overlook and breaks API composability.
- More robust solutions for both multi-trial artifact collection, explicit path handling, and clearer/convenient API for cross-language trial identification would improve usability and predictability, especially for users migrating between Python Optuna and Optuna.jl or running studies with multiple trials.
Problematic multi-trial behavior in
src/artifacts.jl:This function uses
stack()to combine artifact metadata from all trials. The indexing[1, :]is problematic in several cases:get_all_artifact_metadoes not robustly handle studies with more than one trial, since the returned structure is not correctly shaped if trials differ in their artifact metadata.Download artifact path issue:
Here, line
file_path=abspath(file_path) * "$artifact_id.jld2"means the file is always put within a directory, not at the exact path specified, possibly leading to confusion or mismatches when the path is treated as a file rather than a directory.Test coverage does not exercise multiple-trial cases:
The test for
get_all_artifact_metaintest/artifacts.jlonly creates and tests a single trial:This would not catch issues with multiple trials or errors thrown due to stacking in
get_all_artifact_meta.API usability issue for single-trial metadata lookup:
The function signature for
get_all_artifact_meta(study::Study, trial)expects the second argument to be a JuliaTrialobject. However, internally this calls Python'soptuna.artifacts.get_all_artifact_meta(trial; ...), which requires a Python trial object, i.e.,trial.trialfrom a JuliaTrial.This can cause confusion and errors: calling
get_all_artifact_meta(study, trial)with a JuliaTrialworks, but attempting to call it with a Python trial or aTrial.trialobject (as needed by the Python API) will not work as expected.Summary:
get_all_artifact_metaon a study with multiple trials will throw errors, limiting its utility for real-world use cases where studies have many trials.Trial, but the call to the Python backend must get.trial, which is easy to overlook and breaks API composability.