I'm using the manopt package to solve some optimization problems in matlab. The problem is of the form.
problem.cost = @(x) f(x) problem.egrad = @(x) g(x)
After the problem definition, I check the gradient consistency using the following call:
checkgradient(problem)
And then I get the following message:
"The slope should be 2. It appears to be: 1. If it is far from 2, then directional derivatives might be erroneous. The residual should be 0, or very close. Residual: 5.36216e-16. If it is far from 0, then the gradient is not in the tangent space."
Since the algorithm is performing very badly even on very simple examples, I suppose that there is some mistake in the implementation of the gradient. I have some questions about how to interpret the message from the consistency checker.
- What is special about the value 2 for the slope? I have found several other codes on the internet where when executing the gradient consistency checker we get the messages of the form: "The slope should be 2. It appears to be 2.00001" or "The slope should be 2. It appears to be 1.9998". Additionally it looks like the value 2 is a specified a priory in the code of the manopt package. See for instance this code.
- When the slope is far from 2, as in my case, what does that mean? In my case, I don't know how to interpret this. The function I'm trying to optimize is a multilinear polynomial on real variables. Each monomial has 5 variables. But since each variable appears at most once in each monomial, the partial derivative of each variable x is a multilinear polynomial of degree 4 over the other variables.
