A more direct and better variant of New Q-Newton's method Backtracking for m equations in m variables
share
In this paper we apply the ideas of New Q-Newton's method directly to a system of equations, utilising the specialties of the cost function f=||F||^2, where F=(f_1,… ,f_m). The first algorithm proposed here is a modification of Levenberg-Marquardt algorithm, where we prove some new results on global convergence and avoidance of saddle points. The second algorithm proposed here is a modification of New Q-Newton's method Backtracking, where we use the operator ∇ ^2f(x)+δ ||F(x)||^τ instead of ∇ ^2f(x)+δ ||∇ f(x)||^τ. This new version is more suitable than New Q-Newton's method Backtracking itself, while currently has better avoidance of saddle points guarantee than Levenberg-Marquardt algorithms. Also, a general scheme for second order methods for solving systems of equations is proposed. We will also discuss a way to avoid that the limit of the constructed sequence is a solution of H(x)^⊺F(x)=0 but not of F(x)=0.
READ FULL TEXT