Woodgate Algorithm

[banrovegrie]

Hey, this is wrt #170. I had a few doubts. @FanwangM @Ali-Tehrani

[banrovegrie]

Doubt 1

We know that $f$ is defined as follows in the paper. Now, if $F, G \in \mathbb{R}^{n\times m}$ then $F'F \in \mathbb{R}^{m\times m}$. As a result if $n \neq m$ then how will the dimensions of $F'F,\ (E'E)^2GG'$ match? Of course, $E \in \mathbb{R}^{n\times n}$.

[FanwangM]

It seems that $n=m$ in order to make this equation true.

What do you think? @Ali-Tehrani

[Ali-Tehrani]

The formula could have been written better. The trace is invariant under the cyclic permutations (and also transpose), so you can just change the order so that the matrix shapes all match.

Alternately, use the fact that the trace is linear and do (tr(FF) + Tr((E"E)^2 G G") - Tr(E"EQ))/2` separately. The latter is more efficient.

[banrovegrie]

I started working with it by changing the order but taking the traces separately is so much better, thankss.

[banrovegrie]

Doubt 2

To solve the following minimization problem, I used scipy.optimize.minimize. However, I am confused as to how do I consider the fact that $\omega > 0$.

Currently, I have put the bounds as follows and I think (0, None) is considered to be $\geq 0$.

def find_minima(E, DE):
    def func(w):
        return f(E - w * DE)

    # Optimize the function using scipy.optimize.minimize
    # with respect to w > 0.
    w_min = minimize(func, 1, bounds=((0, None),))
    return w_min.x[0]

[FanwangM]

Maybe adding constraints with the following example, https://stackoverflow.com/questions/20075714/scipy-minimize-with-constraints?

[Ali-Tehrani]

I'm fine with replacing zero with a small number 1e-10.

[banrovegrie]

Yep I kept it 1e-9

[banrovegrie]

Doubt 3

The error that we get once the algorithm convergences seems to be dependent on the initial choice of $E_0$ to some extent. Does this mean something is going wrong in the computation or else how should we choose a suitable $E_0$?

[FanwangM]

I am not quite sure about this. But maybe you are right, the algorithm is sensitive to the initialization. Do you have an example?

[banrovegrie]

I think I can fix this with some linear scaling.

[Ali-Tehrani]

If by linear scaling, you mean the alpha in the paper (or the N() function) then I agree with you. Unless I missed it, the code also doesn't seem to scale by alpha at the end of the iteration either.

[FanwangM]

In order to render the equations properly, one will need https://github.com/nschloe/xhub.

[FanwangM]

I have a conference coming tomorrow and I will try to answer your questions tomorrow. Sorry for the delay.

[banrovegrie]

Ayy no no that is fine. You can take a look whenever u are free.