Add an objective merit so line searches serve OptimizationProblems

[ChrisRackauckas-Claude]

Please ignore this PR until it has been reviewed by @ChrisRackauckas. Opened as a draft.

Motivation

Line searches here are currently tied to root finding in two ways:

1. Every CommonSolve.init dispatches on AbstractNonlinearProblem. An OptimizationProblem cannot enter at all — it is not a subtype (OptimizationProblem <: AbstractNonlinearProblem is false, while IntervalNonlinearProblem <: AbstractNonlinearProblem is true).
2. Each algorithm inlines the merit ϕ(α) = ‖F(u + α du)‖²/2 into its own ϕ/ϕdϕ closures.

But the scalar function is the only merit-specific part of any of these algorithms — the bracketing, zoom and interpolation logic is entirely merit-agnostic. Consequently an optimizer that wants Wolfe conditions has to reimplement searches this package already has.

Pointing the existing residual path at ∇f = 0 is not a workaround: ϕ would be ‖∇f‖²/2, which is zero at saddles and maxima too, and deriv_op's dot(fu, jvp_op(du,u,p)) becomes a Hessian-vector product at every trial step.

What this does

• AbstractMerit, with ResidualMerit (‖F‖²/2, ϕ' via a JVP) and ObjectiveMerit (f itself, ϕ' = ⟨∇f, du⟩ — a dot product against a gradient the caller usually already holds).
• MeritEvaluator, built once by init_merit and reused across solve! calls, so a search allocates nothing in steady state.
• init(prob::OptimizationProblem, alg, u) alongside the existing nonlinear entry point, for BackTracking, StrongWolfeLineSearch and GoldenSection.

Two related interface gaps are closed at the same time, both of which cost evaluations today:

• LineSearchSolution gains optional ϕ and dϕ, and the evaluator is left holding the accepted point (ensure_evaluated_at!). Callers previously had to re-evaluate to recover the objective and gradient at the accepted step — a full derivative pass per outer iteration.
• solve! accepts ϕ0/dϕ0 keywords, and set_initial_step! sets the first trial step per call. Every algorithm re-evaluated at α = 0 on entry even though the caller had just computed it. Driving an external L-BFGS, removing that redundancy cut evaluations from 3519 to 1938 on a 45-problem suite — it was close to doubling the gradient count per iteration.

init_merit takes need_deriv, so GoldenSection stays derivative-free instead of being forced to build a Jacobian operator it never uses.

Compatibility

• LineSearchSolution(step_size, retcode) still constructs; the new fields default to nothing.
• LiFukushimaLineSearch and RobustNonMonotoneLineSearch are untouched — their merits (‖F‖, ‖F‖^n_exp) are residual-specific by design and baked into their acceptance tests.
• Existing Core and LineSearchesJL groups pass unchanged (229 tests). Runic clean.

Tests

test/objective_merit_tests.jl covers the objective entry point for each converted algorithm, the derivative-free path, ϕ/dϕ reporting and cache reuse, both gradient arities (2-arg instantiated and 3-arg raw), a clear error when no gradient is available, zero allocations in solve!, and Float32/BigFloat.

The sharpest check states one problem both ways — F(u) = 0 with merit ‖F‖²/2, and the objective f(u) = ‖F(u)‖²/2 — and asserts the two merits produce identical step sizes, since ϕ is literally the same scalar function.

🤖 Generated with Claude Code