Skip to content

[BUG] cuopt_cli MIP emits stale suboptimal incumbent when optimal solution found quickly #1488

Description

@caelunshun

Describe the bug

I'm finding that for a fairly small MILP problem that I hand to cuopt_cli via the LP format reader, cuopt is claiming to have found an optimal solution ("Optimal solution found"), but then writes a different solution (with worse objective value) to its solution file. I suspect that the small problem size/very fast convergence time plays a role here, e.g. a race condition in the solution tracking somewhere.

Steps/Code to reproduce bug

Here is the LP file I am able to reproduce the bug with. Save it as test.lp. When running cuopt_cli test.lp --solution-file sol.txt, I get the following on stdout:

cuopt_cli output (also includes version and environment info)
Setting CUDA_MODULE_LOADING to EAGER
Setting parameter solution_file to sol.txt
Reading file test.lp
Read file test.lp in 0.00 seconds
cuOpt version: 26.6.0, git hash: df5fe0d1, host arch: x86_64, device archs: 70-real,75-real,80-real,86-real,90a-real,100f-real,120a-real,120
CPU: Intel(R) Core(TM) Ultra 7 265H, threads (physical/logical): 16/16, RAM: 13.15 GiB
CUDA 12.9, device: NVIDIA RTX PRO 2000 Blackwell Generation Laptop GPU (ID 0), VRAM: 7.96 GiB
CUDA device UUID: <redacted>

Solving a problem with 210 constraints, 80 variables (26 integers), and 496 nonzeros
Problem scaling:
Objective coefficents range:          [4e-02, 1e+00]
Constraint matrix coefficients range: [1e+00, 2e+00]
Constraint rhs / bounds range:        [5e-01, 3e+00]
Variable bounds range:                [0e+00, 2e+00]

MIP row scaling completed
New solution from early primal heuristics (CPUFJ). Objective +1.000000e+00. Time 0.01
New solution from early primal heuristics (CPUFJ). Objective +9.619982e-01. Time 0.01
Original problem: 210 constraints, 80 variables, 496 nonzeros
Calling Papilo presolver (git hash 741a2b9c)
New solution from early primal heuristics (CPUFJ). Objective +5.182279e-01. Time 0.01
Presolve status: reduced the problem
Presolve removed: 102 constraints, 27 variables, 218 nonzeros
Presolved problem: 108 constraints, 53 variables (18 integer), 278 nonzeros
Papilo presolve time: 0.02
Objective offset 0.000000 scaling_factor 1.000000
Model fingerprint: 0x459a86b7
Starting cuOpt presolve
Running probing cache with 15 tasks
After cuOpt presolve: 108 constraints, 53 variables, objective offset 0.000000.
cuOpt presolve time: 0.03
Graph construction time 0.000059
Automorphism group size 1*10^0, 0 dejavu generators (0 move variables)
Dejavu time 0.000084
Projected 0 generators onto 53 binary variables (0 skipped non-binary), 0 stored
Total symmetry detection time 0.000153
No exploitable symmetry found (0 generators, 0 non-trivial orbits, max orbit size 0)
Probing implied bounds: 8 zero entries, 8 one entries
Reduced cost strengthening enabled: 2
150 variable upper bounds in 0.00 seconds
70 variable lower bounds in 0.00 seconds

Solving LP root relaxation in concurrent mode
Skipping column scaling
Dual Simplex Phase 1
Dual feasible solution found.
Dual Simplex Phase 2
 Iter     Objective           Num Inf.  Sum Inf.     Perturb  Time
    0 +4.2107386559337912e-01      12 1.80000000e+01 0.00e+00 0.10
    1 +4.2107386559337912e-01      12 1.80000000e+01 0.00e+00 0.10


Root relaxation solution found in 40 iterations and 0.00s by Dual Simplex
Root relaxation objective +4.21073866e-01


 | Explored | Unexplored |    Objective    |     Bound     | IntInf | Depth | Iter/Node | Gap    |  Time  |
           0            0    +5.182279e-01    +4.358725e-01       14      0   7.2e+01      15.9%      0.10
           0            0    +5.182279e-01    +4.531376e-01       11      0   8.8e+01      12.6%      0.10
           0            0    +5.182279e-01    +4.605369e-01       12      0   9.8e+01      11.1%      0.10
           0            0    +5.182279e-01    +4.664564e-01       12      0   1.2e+02      10.0%      0.10
           0            0    +5.182279e-01    +4.823826e-01       12      0   1.4e+02       6.9%      0.10
           0            0    +5.182279e-01    +4.885996e-01        8      0   1.6e+02       5.7%      0.10
           0            0    +5.182279e-01    +4.901342e-01        9      0   1.8e+02       5.4%      0.10
           0            0    +5.182279e-01    +4.956152e-01       10      0   1.9e+02       4.4%      0.10
           0            0    +5.182279e-01    +4.956152e-01        3      0   2.0e+02       4.4%      0.11
H                            +5.000000e-01    +4.956152e-01                                 0.9%      0.11
           0            0    +5.000000e-01    +5.000000e-01        9      0   2.0e+02      0.00%      0.11
Explored 0 nodes in 0.11s.
Absolute Gap 0.000000e+00 Objective 5.0000000000000000e-01 Lower Bound 5.0000000000000000e-01
Optimal solution found.
Solution objective: 0.518228 , relative_mip_gap 0.035174 solution_bound 0.500000 presolve_time 0.047966 total_solve_time 0.114318 max constraint violation 0.000000 max int violation 0.000000 max var bounds violation 0.000000 nodes 0 simplex_iterations 205
Writing solution to file sol.txt

Notice the line "Optimal solution found," the absolute gap of 0.0, and the objective of 0.5. These are all consistent with the solver having found a global optimum. However, the subsequent line reads "Solution objective: 0.518228," which clearly corresponds to a different, suboptimal solution (namely the incumbent solution from a few ms earlier during the solve).

The contents of the sol.txt output are below. Here we see again this suboptimal objective value. The status of "FeasibleFound" instead of "Optimal" further suggests this solution is stale.

sol.txt contents
# Status: FeasibleFound
# Objective value: 0.518227920211888016
var46 0.5
var79 0
var61 0.961898212776501449
var59 0.5
var77 0.0381017872234986621
var75 0.0381017872234976629
var0 1
var1 0
var2 0.5
var3 0.5
var4 1
var5 1
var6 0.5
var7 0.5
var8 0
var9 0.5
var10 0.5
var11 1
var12 0
var13 1
var14 1
var15 0
var16 1
var17 0
var18 1
var19 0.5
var20 0
var21 0.5
var22 0
var23 1
var24 0
var25 0.961898212776501449
var26 0.0381017872234985511
var27 1
var28 1
var29 0.961898212776501449
var30 0.0381017872234986621
var31 0
var32 0.961898212776501449
var33 0.961898212776501449
var34 1
var35 0
var36 1
var37 1
var38 0
var39 1
var40 0
var41 1
var42 0.961898212776501449
var43 0
var44 0.0381017872234986621
var45 0
var47 0.5
var48 0
var49 0
var50 0
var51 1.5
var52 1
var53 1
var54 1
var55 0
var56 0
var57 2
var58 1
var60 0
var62 0
var63 1
var64 0
var65 0.961898212776502337
var66 1
var67 1.03810178722349766
var68 1
var69 1
var70 0
var71 0
var72 0
var73 2
var74 1
var76 0
var78 0

Expected behavior

The solver should emit the optimal solution that it found, i.e. the solution having objective 0.5 for the problem above.

Metadata

Metadata

Assignees

Labels

awaiting responseThis expects a response from maintainer or contributor depending on who requested in last comment.bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions