Required new formulation for the Multi-power law

[maximejay]

Hello,

To pick up on the issue #450, I would like to suggest a new formulation of the multi-power law.

The current multi-power law might be difficult to calibrate and first attempts (last year) to calibrate the power failed (no convergence).

Here is the current law:

$\theta(x)=\rho_0 + \rho_1 \mathcal{D}^{\rho_2}(x) $

If I understand well, the descriptor are values normalized between [0:1] before calibration. The power $\rho_2$ of the multi-power-law is calibrated such that $\rho_2 \in R$. Then the sigmoïde is applied.

Here is a plot of the multi-power law for different value of $\rho_2$, with $\rho_0=0$ and $\rho_1=1$.

We can see that these functions increases or decreases depending the value of $rho_2$ : if $rho_2<0$, $\theta(x)$ decrease, if $rho_2>0$, $\theta(x)$ increase. Moreover, these functions are not normalized between [0:1] anymore.

Thus, I suggest to use a different formulation:

$\theta(x)=\rho_0 + \rho_1 \mathcal{D}^{\beta}(x) $
with,
$\beta=10^{\rho_2}$

This lead to a more consistent variation of the descriptor in the interval [0:1]:

[ngocbao1022]

Hello Maxime,

I'm sorry in advance that maintaining this code and resolving issues is not exactly in my scope of work but I'm really interested by this bug (it potentially impacts my next steps).

As you stated, the current multi-power mapping is difficult to calibrate, so I assume this proposition is made to make it easier to calibrate. So I have 2 questions:

1. How exactly the "increase and decrease depending on rho2" problem makes it harder to calibrate? It is difficult to understand since there is no explaination on how it affects calibration. Can you elaborate please?
2. Do you have an example where current law fails to calibrate while the proposed law works well? How better does it converge (in term of time, number of iterations)?

[maximejay]

Hello,

As far as I Know, calibrating the power exponent has not be scrupulously tested/tried. One attempt was made by Didier (hydris) last years (or may be earlier). The optimization failed (no convergence, no minimization). After this test we discussed a little bit about this failed, and we discover that the power law applied to the descriptor is not properly shaped (in my opinion).

1. These curves suggest that there will be some big "jump" in the cost when the sign of the exponent changes. The differentiation with tapenade is probably not valid in that case (asymptote). Moreover, the sensibility of the exponent respect to the descriptor is not homogeneous : higher sensibility for low values of the descriptor, lower sensibility for high value (changing the power will cause big change on the parameter where the descriptors are low, but almost no change for higher value of the descriptors).
2. I have no example and I never test this formulae. It is just a suggestion. But with this new formulae, there is no asymptote around 0 with infinite value.

It will be needful to test both formula :

1. Check how the descriptors are normalized (if it is the case), and adjust the formulae
2. make manual gradient test for a power different of 1 (for the current formulae) and different of 0 (for the suggested formulae) to validate the differentiation.
3. try the optimization with both formula and compare the results

[nghi-truyen]

Hello,
Thanks @maximejay for the suggestion, thanks @ngocbao1022 for the comment.
First, I would like to clarify that the exponent value is bounded between 0.5 and 2 in SMASH. If I understand well, the new formulation (D^(10^rho)) introduce the exponents varied from 1 to 10?
The descriptors are normalized using min-max scaling, so they are bounded between 0 and 1. The output has no bound constraint, but we apply the sigmoid function and then scale it to the physical bounds.
So I agree with both of you: a comparison between the two formulations is necessary to decide whether we should add the new formulation or replace the current one. Moreover, a benchmark on a large dataset is also necessary to evaluate the methods.

[maximejay]

Hello,
In the new formulae the exponent will vary theoretically from -inf to +inf (no bounds required), and 0 correspond to a linear formulation.

I was not aware of bounds between 0.5 and 2 for this coefficient. In that case, the shape of the curves are more equivalent with my suggestion.

But with the new formulation, you can describe all the field: