Runner: use new conda release for HiGHS HiPO

[siddharth-krishna]

HiGHS has released a distribution including HiPO on their conda dev channel:

conda install -c conda-forge/label/dev highspy

• Remove the code added by Runner: add support to build & run HiGHS HiPO solver #388 to build HiPO from source and call the binary
• Instead, for solvers highs-hipo and highs-ipm use the above conda distribution to install
• Modify the code in run_solver.py to choose the hipo or ipx algorithm via linopy
• Test on the following benchmark to ensure that we see a similar performance for the highs variants:

machine-type: c4-standard-8
zone: us-central1-a
years:
- 2025
benchmarks:
  pypsa-de-elec:
    Problem class: LP
    Sizes:
    - Name: 6-1h
      Size: L
      URL: https://storage.googleapis.com/solver-benchmarks/pypsa-de-tests/pypsa-de-elec_6_1h.mps.gz
solver: highs-hipo highs-ipm highs
timeout_seconds: 86400

[MaykThewessen]

HiGHS HiPO Python Build Experience & Benchmark Results

We've been running HiPO benchmarks on our PyPSA DC-OPF model (Netherlands 380/220/150/110kV grid, ~200 buses, ~600k constraints per weekly chunk). Sharing our build experience and findings since this issue is working toward the same goal.

Building HiPO for Python (macOS ARM64)

PyPI highspy 1.13.1 does not include HiPO — the upstream CMakeLists.txt explicitly blocks it:

if (HIPO AND PYTHON_BUILD_SETUP)
  message(ERROR "HIPO is only available in C++. Not yet supported in Python.")
endif()

We patched this based on the approach in PR #2756:

1. CMakeLists.txt: Remove the Python+HiPO block, keep set(HIPO ON)
2. cmake/sources-python.cmake: Add HiPO source files (hipo/ipm/*.cpp, hipo/factorise/*.cpp, etc.)
3. Missing dependencies: Had to add PreProcess.cpp and OrderingPrint.h that weren't in PR #2756's source lists
4. Build: pip install . --config-settings=cmake.define.HIPO=ON

On macOS, HiPO uses Apple Accelerate for BLAS and METIS/AMD from extern/. Build succeeds but needs the extra files.

Benchmark Results (168h DC-OPF, 10 threads, M-series Mac)

Solver Config	Backend Reported	Solve Time	Objective
simplex	simplex-pami	45.9s	254,742,757
ipm (IPX, PyPI stock)	ipx	51.9s	254,752,916
hipo (custom build)	simplex-pami	79.5s	254,742,757

Key finding: HiPO appears to fall back to simplex-pami rather than actually running the parallel IPM. The backend string simplex-pami and the objective matching simplex exactly (rather than IPX's slightly different solution) confirms this. This may be because:

• Our LP (with run_crossover: 'off') triggers a fallback path
• The HiPO solver option is accepted but the actual solver selection logic routes to simplex

For 336h (2 weeks):

Solver Config	Solve Time	Objective
simplex	166.6s	766,210,444
ipm (IPX)	197.4s	766,243,099
hipo	244.9s	766,210,444

Other Findings

• Presolve is critical: IPM without presolve takes 693s vs 51.9s with presolve (13x slowdown)
• Thread scaling is flat for IPX: 1 thread (51.9s) ≈ 4 threads (50.7s) ≈ 10 threads (51.9s) — IPX is serial
• simplex_scale_strategy: 4 (max-value scaling) gives marginal improvement for IPM on our badly-scaled DC-OPF (8 orders of magnitude coefficient range)
• Tolerance sweep: 1e-4, 1e-6, 1e-8 all converge in similar time for IPX

For Your Runner

If you're building from conda dev channel, verify the actual backend used. Check h.getInfoValue('solver_name') or similar — HiPO may silently fall back to simplex depending on problem type and options. Our MPS exports are available if useful for testing.

[MaykThewessen]

Benchmark Charts — PSA_MEE DC-OPF (Netherlands Grid)

Follow-up to our earlier comment with detailed benchmark results. We have PNG charts available — happy to share them directly if useful for the project.

Chart 1: Solver Comparison (168h / 336h / 672h)

Solve Time vs Problem Size:

Solver	168h	336h	672h
simplex	46s	167s	~680s
ipm-stock (IPX)	52s	197s	~700s
hipo (custom build)	80s	245s	~780s
ipm (PyPI)	52s	197s	~700s

Key observations:

• Simplex wins on all sizes for this DC-OPF problem class
• HiPO falls back to simplex-pami — not actually running parallel IPM (see objectives match simplex exactly)
• IPX is serial: 1→4→10 threads shows 1.0x speedup (flat line on thread scaling chart)
• Peak RAM: ~190 MB at 168h, scales to ~2.6 GB at 672h (simplex highest)

CPU Utilization:

• Simplex: 87% → 375% → 569% (good multi-thread scaling)
• HiPO: 87% → 264% → 589% (moderate scaling)
• IPM/IPX: 100% across all sizes (serial — no thread utilization)

Cost of Energy (solver accuracy check):

• All solvers converge to 15.19 EUR/MWh at 168h
• Slight divergence at 672h: simplex 17.98, IPM 0.86 EUR/MWh — likely tolerance-related

Chart 2: Parameter Sensitivity (IPM-stock 168h)

All configurations produce identical cost (15.19 EUR/MWh):

Configuration	Solve Time
ipm-stock-168h (baseline)	49s
ipm-stock-1t (1 thread)	49s
ipm-stock-4t (4 threads)	48s
ipm-stock-crossover-on	61s
ipm-stock-tol-1e6	54s
ipm-stock-tol-1e8	56s
ipm-stock-scale-2 (equilibration)	48s
ipm-stock-no-presolve	693s (13.5× slower!)

Presolve is the dominant factor — disabling it causes a 13.5× slowdown. Thread count, tolerance, and scaling strategy have marginal impact on IPX.

Chart 3: Thread Scaling (IPM)

IPX shows zero parallel speedup: 49s (1 thread) → 48s (4 threads) = 1.0× speedup vs ideal 4.0×. This confirms IPX is fundamentally serial, making HiPO's parallel factorization potentially very valuable — if it actually runs.

Hardware

• Apple M-series (ARM64), 10 logical cores
• 16 GB RAM
• macOS 15.3

We can export our 168h/336h/672h DC-OPF problems as MPS files if useful for the benchmark collection.

[finozzifa]

Hi @siddharth-krishna,
a few questions on the issue task

Remove the code added by Runner: add support to build & run HiGHS HiPO solver #388 to build HiPO from source and call the binary

Do I need to remove the following code snippets? Please see the list below:

• infrastructure/startup-script.sh: remove the lines 58 through 105
• runner/run_benchmarks.py: remove get_highs_hipo_version

Is my understanding correct? Thanks!

[siddharth-krishna]

@MaykThewessen thank you for sharing your experiences, and apologies for the slow reply as we were preparing our v2 release, call for benchmarks, and a blog post detailing our experience with HiPO. A few thoughts:

• Yes, we are aware that highs releases on pypi do not have hipo. Hence our blog post study built it from source. The highs releases page now has pre-built binaries including hipo (the ones ending -apache). Perhaps that's worth trying!
• I'm surprised that simplex beats hipo on your benchmark models. Our study using PyPSA models showed a dramatic speedup of hipo over both simplex and ipx, especially as the size grows into the millions. We ran our benchmark on a Debian VM with 8vCPUs, and built from source -- perhaps one of those explain the difference? Did you by chance ask the highs team for their thoughts?
• Alternatively, perhaps this is to do with model / LP problem characteristics. If you could export MPS files and contribute them, that would be an interesting addition to this platform. Thank you!

@finozzifa thanks for looking into this again.

• Yep, remove the lines that build HiPO from source
• Yep, remove get_highs_hipo_version and ensure that get_conda_package_versions also works to get the version of highs-hipo.

Essentially, we need a way to run hipo via the conda installation. Since it's probably unlikely that we can have an installation of the highs package from the main channel as well as the dev channel in the same conda env, you might have to make a new one (2026 perhaps?) for the dev channel package that includes hipo. Let the 2025 env continue to use the same package versions in case we need to run more benchmarks. Does that make sense? Happy to discuss :)