Currently the default optimizer is with the Ax agent, which uses a global optimization method and requires uniform sampling of the domain to prepare conditional and Gibbs sampling. This is powerful when there is not prior knowledge of the space (black box), but reduces relative performance for when prior assumptions can be made.
This includes priors like:
- we already start in a "good enough" subspace (and bounds are fuzzy)
- we can make claims about the conditional distribution before starting (eg Gaussian centered around start point, or any other skewed/non isometric dist)
- Only a subset of parameters given actually affect the objective.
Ultimately, I suggest we add a gradient based optimizer to the default utilities so that we can target the use case where:
- ||solution-start|| << search space diameter
- the space is locally differentiable and a local minima is acceptable
In other words, the optimal solution is a small perturbation of a high-dimensional starting point. This is good for problems associated with drift and optimizing a distribution quality rather than the distribution itself (ie reduce skew, make more gaussian, improve collimation ...)
Currently the default optimizer is with the Ax agent, which uses a global optimization method and requires uniform sampling of the domain to prepare conditional and Gibbs sampling. This is powerful when there is not prior knowledge of the space (black box), but reduces relative performance for when prior assumptions can be made.
This includes priors like:
Ultimately, I suggest we add a gradient based optimizer to the default utilities so that we can target the use case where:
In other words, the optimal solution is a small perturbation of a high-dimensional starting point. This is good for problems associated with drift and optimizing a distribution quality rather than the distribution itself (ie reduce skew, make more gaussian, improve collimation ...)