`compute_conflict!` silently aborts the IIS

[resmaeilbeigi]

Summary

MOI.optimize! leaves an optnode callback registered when it returns: the null one for MIPs
(L2914-L2926), or default_moi_callback whenever a lazy/user-cut/heuristic callback is
set (L2903-L2913, no _is_mip guard). Its wrapper calls reenable_sigint
(MOI_callbacks.jl#L43), which is balanced during the solve because XPRSoptimize runs
inside disable_sigint (L2946-L2949).

_getfirstiis has no such region: XPRSiisfirst is called bare (L4304-L4307) with that
callback still attached. The callback does fire during the IIS search (measured below), so
XPRSiisfirst runs at least one branch-and-bound search internally, although FICO does not
document that. reenable_sigint therefore throws
ErrorException("sigatomic_end called in non-sigatomic region"), and the wrapper stores it and
stops the solver with XPRSinterrupt(cb_data.model, XPRS_STOP_USER) (L44-L51). The IIS
search is abandoned and comes back as status 3.

compute_conflict! then returns normally, warns about "invalid bounds" on a model whose
bounds are all valid, and reports NO_CONFLICT_FOUND for a model with a genuine constraint
conflict. The exception is never surfaced, because _check_cb_exception is called only from
MOI.optimize! (L2951) and the next MOI.optimize! clears model.cb_exception in
_pre_solve_reset (L2855-L2864).

Introduced in v0.18.0 by #307, still present on master: 9c15421 is both master's head and
the commit tag v0.18.1 points at.

Separately, and only seen once: a large infeasible MIP segfaulted inside libxprs, on the
first XPRSgetrows after compute_conflict!. No log of that run was kept, and it differed
from the environment below: it ran Xpress.jl v0.18.0, Xpress 9.8.1 (libxprs.so.46), and
aarch64-linux-gnu. The v0.18.0 defect noted under Environment is the more likely cause there,
in which case compute_conflict! threw rather than returned and the exception was swallowed
by the calling code. It is not reproduced here and should be treated as unrelated to the
reproducer below.

Environment

Xpress.jl v0.18.1, Xpress 9.9.1 (libxprs.so.47), MathOptInterface v1.51.1, JuMP v1.30.1,
Julia 1.12.6, x86_64-linux-gnu. Line references are to 9c15421 (master's head at the time of
writing, which is also the v0.18.1 tag).

On this code path v0.18.0 differs only in names (getfirstiis, check_cb_exception) and in
one real defect: its catch branch calls callback_exception(model, cb_data, ex), which
evaluates cb.callback_data.model. CallbackData has only a model field, so the
.callback_data access throws FieldError and XPRSinterrupt is never reached. That second
exception unwinds out of the _cboptnode cfunction into the nearest enclosing Julia handler,
so on v0.18.0 compute_conflict! throws instead of returning quietly; the Actual output below
is v0.18.1 behaviour. #325 fixed it by inlining callback_exception at the catch site and
using cb_data.model (MOI_callbacks.jl#L44-L51), and added test_error_in_callback as a
regression test. (The identical-looking cb.callback_data.model that remains at
MOI_callbacks.jl#L183 is a different path, _throw_if_invalid_state, where cb really is an
MOI.UserCut/LazyConstraint/HeuristicSolution and does have that field.) v0.18.1 also
carries changes unrelated to this bug: a DualObjectiveValue fix, NLPLOG support and a style
refactor.

Reproducer

using JuMP
import Xpress
import MathOptInterface as MOI

model = direct_model(Xpress.Optimizer())
set_attribute(model, "OUTPUTLOG", 0)
# These four controls are what make this tiny example reproduce: they force the
# IIS sub-MILP into a tree search, which is what makes the optnode callback
# fire. Without them the same model presolves to an answer without visiting a
# node and prints the Expected output below, so do not drop them.
for k in ("PRESOLVE", "MIPPRESOLVE", "HEURSTRATEGY", "CUTSTRATEGY")
    set_attribute(model, k, 0)
end

n = 10
@variable(model, x[1:n], Bin)
# Even coefficients, odd right-hand side: LP feasible, integer infeasible.
@constraint(model, c, sum(2 * i * x[i] for i in 1:n) == 2 * sum(1:n) - 1)

optimize!(model)
@assert termination_status(model) == INFEASIBLE

compute_conflict!(model)

@show backend(model).cb_exception
@show backend(model).conflict.stat
@show MOI.get(model, MOI.ConflictStatus())
@show MOI.get(backend(model), MOI.ConstraintConflictStatus(), index(c))

Actual

┌ Warning: Xpress can't find IIS with invalid bounds, the constraints that keep the model infeasible can't be found, only the infeasible bounds will be available
└ @ Xpress .../Xpress/src/MOI/MOI_wrapper.jl:4312
(backend(model)).cb_exception = ErrorException("sigatomic_end called in non-sigatomic region")
(backend(model)).conflict.stat = 3
MOI.get(model, MOI.ConflictStatus()) = MathOptInterface.NO_CONFLICT_FOUND
MOI.get(backend(model), MOI.ConstraintConflictStatus(), index(c)) = MathOptInterface.NOT_IN_CONFLICT

Constraint c is the entire cause of the infeasibility and no bound is violated, so the
warning and both statuses are wrong.

Expected (with the fix below)

(backend(model)).cb_exception = nothing
(backend(model)).conflict.stat = 0
MOI.get(model, MOI.ConflictStatus()) = MathOptInterface.CONFLICT_FOUND
MOI.get(backend(model), MOI.ConstraintConflictStatus(), index(c)) = MathOptInterface.IN_CONFLICT

Cause

disable_sigint/reenable_sigint are a counter: Base.reenable_sigint(() -> nothing) alone
throws sigatomic_end called in non-sigatomic region. Nothing deregisters the callback when
optimize! returns, since _pre_solve_reset does not touch it, so the callback registered
through XPRSaddcboptnode (MOI_callbacks.jl#L56-L61) is still attached when
XPRSiisfirst runs outside any sigatomic region.

The callback body never runs during the IIS: reenable_sigint(f) calls sigatomic_end()
before f, so it throws first. A counting callback registered through
Xpress.CallbackFunction records 10 invocations during optimize! (21 nodes) and 0 during
compute_conflict!, while cb_exception is set.

status_code[] comes back as 3, read as a timeout (# 2 = error, 3 = timeout) although
nothing timed out. That maps onto the "invalid bounds" warning and a bounds-only IISData,
which for this model is empty (colnumber == 0, miiscol == Cint[]), so not even the
infeasible bounds the warning promises are available. MOI.get(::ConflictStatus) then returns
NO_CONFLICT_FOUND because no bound conflict exists (L4405-L4421).

The test_conflict_* tests miss it: they are either pure LPs, where no callback is
registered, or one- and two-variable MIPs whose IIS is resolved without the sub-MILP visiting
a node. MathOptInterface's own test_solve_conflict_* tests, run through MOI.Test.runtests,
miss it for the same two reasons.

Possible Fix

Mirror what MOI.optimize! already does, and surface the exception:

 function _getfirstiis(model::Optimizer)
     status_code = Ref{Cint}(0)
-    ret = XPRSiisfirst(model, 1, status_code)
+    # The optnode callback that `MOI.optimize!` leaves registered calls
+    # `reenable_sigint`, so the IIS solve must run inside a sigatomic region,
+    # exactly like `XPRSoptimize` does in `MOI.optimize!`.
+    ret = disable_sigint() do
+        return XPRSiisfirst(model, 1, status_code)
+    end
+    _check_cb_exception(model)
     _check(model, ret)
     if status_code[] == 1  # The problem is actually feasible.

disable_sigint forwards the return value, so ret is unchanged. _check_cb_exception goes
before _check, so a return code caused by the interrupt cannot mask the real cause;
MOI.optimize! likewise discards the return code of XPRSoptimize. The IIS also becomes
CTRL+C interruptible whenever a callback is attached; with none attached, as for an LP, it stays
uninterruptible either way, since Julia defers sigint for the whole ccall. Two considerations:

• The invariant then lives in two places. It is better held in one helper used by both
  call sites, so a future internally-solving call cannot reintroduce this. XPRSoptimize and
  XPRSiisfirst are the only two such calls today:

  function _call_xpress_solver(f::F, model::Optimizer) where {F<:Function}
      ret = disable_sigint(f)
      _check_cb_exception(model)
      return ret
  end

• The sigatomic region makes the callback body actually run during the IIS. Harmless for
  the null callback, but default_moi_callback (L2903-L2913) is left registered too,
  and dispatches the user's heuristic, user-cut and lazy callbacks
  (MOI_callbacks.jl#L130-L142); after this change those would run against IIS
  subproblems the user never sees. Deregistering the callback at the end of MOI.optimize!
  avoids that, but costs the CTRL+C support Add support for CTRL+C during the solve #307 added, since with no callback attached Julia
  disables sigint for the whole ccall. Doing both keeps interruptibility without exposing
  user callbacks to subproblems.

Related defect

_getfirstiis maps status 2 (error) and status 3 (timeout) onto the same "invalid bounds"
warning plus a bounds-only IISData. Through the MOI API a caller therefore cannot tell an IIS
error from an IIS timeout, and a failed search surfaces either as NO_CONFLICT_FOUND on a model
that does have a conflict, or, if the model happens to contain any l > u column, as an
ordinary CONFLICT_FOUND (L4405-L4421) indistinguishable from a real IIS. (NO_CONFLICT_EXISTS,
status 1, is reported distinctly, so the confusion is between "the search failed" and "the
search succeeded", not between failure and feasibility.) It is reachable on its own through a
genuine time-limit stop (TIMELIMIT; MAXTIME is deprecated as of Xpress 9.0), and it is what
makes the bug above silent. At minimum the warning should not claim invalid bounds when no
bound is crossed, and status 2 and 3 should be reported distinctly.

[odow]

Closed by #333 and #335

[odow]

Fixed and a new release tagged.

The test_conflict_* tests miss it: they are either pure LPs, where no callback is registered, or one- and two-variable MIPs whose IIS is resolved without the sub-MILP visiting a node. MathOptInterface's own test_solve_conflict_* tests, run through MOI.Test.runtests, miss it for the same two reasons.

This is definitely an issue to think about more generally. We tend to write small trivial tests, but then we miss things like this that only occur on larger more complicated models.