Add Hager-Zhang and More-Thuente line searches

[ChrisRackauckas-Claude]

Please ignore this PR until it has been reviewed by @ChrisRackauckas. Opened as a draft.

Stacked on #86. Review that first — this branch is built on it and its diff therefore includes #86's commit. GitHub will not let me base against a branch that lives only on my fork; once #86 merges, this diff reduces to the two new algorithms alone. Review only the second commit (Add Hager-Zhang and More-Thuente line searches).

What this adds

HagerZhangLineSearch and MoreThuenteLineSearch. Neither existed here; the only route to Hager–Zhang was the LineSearchesJL wrapper.

Both are written against the AbstractMerit abstraction from #86, so one implementation serves NonlinearProblems and OptimizationProblems. A test states a problem both ways and asserts the two merits produce identical step sizes.

Why Hager–Zhang matters

It accepts a step satisfying either the original Wolfe conditions or the approximate Wolfe conditions

σ ϕ'(0) ≤ ϕ'(α) ≤ (2δ − 1) ϕ'(0),    ϕ(α) ≤ ϕ(0) + ε_k

The approximate form stays testable once ϕ(α) − ϕ(0) has fallen to roundoff — exactly where the original conditions become numerically unsatisfiable, near a minimizer.

The regression test pins this to a case where it is unambiguous rather than statistical: f(x) = Σ log(cosh(xᵢ − 1)) underflows to exactly 0.0 for |xᵢ − 1| ≲ 1e-8, while ∇f = tanh(xᵢ − 1) is still nonzero. ϕ carries no detectable decrease at all, so a strict strong-Wolfe search stalls above the gradient floor while Hager–Zhang still accepts.

MoreThuenteLineSearch follows MINPACK-2 dcsrch/dcstep, minimizing ψ(α) = ϕ(α) − ϕ(0) − ftol·α·ϕ'(0) during its first stage.

One subtle detail worth review

Both judge bracket collapse relative to the bracket's own magnitude, not against an absolute eps(T):

w = b.α - a.α
!(w > eps(T) * max(abs(b.α), floatmin(T)))

With a large gradient the optimal step is legitimately far below machine epsilon. On the Moré–Garbow–Hillstrom variably_dimensioned problem at n = 1000 it is ~1e-22, and an absolute floor at 2.2e-16 ends the search before it begins — the solver fails on iteration 1 without moving. This also removes spurious line-search failures well away from that extreme.

Verification

Driving an external L-BFGS over 45 unconstrained problems (Moré–Garbow–Hillstrom plus scalable and applied ones), against Optim.jl with the identical fused value+gradient closure, starting points, per-problem gradient tolerance and caps:

line search	solved (LineSearch.jl / Optim)	evaluations on common set
Hager–Zhang	36 / 34	1938 vs 1988
Moré–Thuente	34 / 31	1738 vs 1630
StrongWolfe	36 / 27	1848 vs 1805

Zero allocations per solve at every problem size, and 1.1×–2.4× faster wall clock with analytic gradients. A solve counts only if it reaches the tolerance and an objective as good as the best any solver found — a small gradient alone is not convergence, since exp-based objectives can underflow to a flat plateau where ∇f is identically zero far from any minimizer.

Existing Core and LineSearchesJL groups pass unchanged. Runic clean.

🤖 Generated with Claude Code