fix: LoadManager::run() fee adjustment block outside while loop silently disables local fee protection

[rippleitinnz]

Summary

The fee adjustment block in LoadManager::run() is placed outside the while loop. As a result, lowerLocalFee() and raiseLocalFee() are called exactly once - when stop_ is set during shutdown - rather than every second during normal operation. localTxnLoadFee_ (reported as load_factor_local in server_info) is therefore never adjusted by local job queue load during normal operation, silently disabling the node's primary self-protection mechanism against JQ saturation.

Bug identified by Nik Bougalis (@nbougalis).

---

Background

LoadManager::run() runs a dedicated thread that ticks every second. Its two responsibilities are:

1. Stall detection - monitor the heartbeat timer and log/abort if the server stalls
2. Local fee adjustment - call raiseLocalFee() when the job queue is overloaded, lowerLocalFee() otherwise, and call reportFeeChange() if the fee changed

The local fee (localTxnLoadFee_ in LoadFeeTrack) is a node-level protection mechanism independent of the open ledger fee escalation driven by TxQ. When a node's job queue is saturated, raiseLocalFee() increases localTxnLoadFee_, which causes Transactor::minimumFee() (via scaleFeeLoad()) to require a higher fee from incoming transactions. Transactions that do not meet the elevated minimum are rejected with telINSUF_FEE_P before consuming any network resources.

This mechanism is entirely separate from load_factor_fee_escalation, which responds to ledger capacity (transaction volume) and was not affected by this bug.

---

Root Cause

In src/xrpld/app/main/LoadManager.cpp, the fee adjustment block sits after the closing brace of the while loop:

void
LoadManager::run()
{
    // ...
    while (true)
    {
        t += 1s;
    std::unique_lock sl(mutex_);
    if (cv_.wait_until(sl, t, [this] { return stop_; }))
        break;       // <-- exits loop on shutdown

    // Stall detection logic ...

}   // <-- while loop ends here

// Fee adjustment block — only reached after stop_ is set (shutdown)
bool change = false;
if (app_.getJobQueue().isOverloaded())
{
    JLOG(journal_.info()) << "Raising local fee (JQ overload): "
                          << app_.getJobQueue().getJson(0);
    change = app_.getFeeTrack().raiseLocalFee();
}
else
{
    change = app_.getFeeTrack().lowerLocalFee();
}

if (change)
{
    // VFALCO TODO replace this with a Listener / observer and
    // subscribe in NetworkOPs or Application.
    app_.getOPs().reportFeeChange();
}

}

cv_.wait_until() returns true when stop_ is set, which causes break to exit the loop. The fee adjustment block that follows is therefore only executed once - on shutdown - and never during normal operation.

---

Effect

• load_factor_local never appears in server_info during normal operation (it is only reported when localTxnLoadFee_ exceeds the baseline of 256)
• raiseLocalFee() never fires regardless of job queue pressure
• lowerLocalFee() never fires, so an elevated localTxnLoadFee_ is never decayed back to baseline
• Incoming transactions are never rejected with telINSUF_FEE_P due to local load, regardless of how saturated the job queue is
• The node silently accepts work at base fee while under genuine stress, compounding the overload

---

Fix

Move the fee adjustment block inside the while loop, after the stall detection block, before the closing brace:

while (true)
{
    t += 1s;
std::unique_lock sl(mutex_);
if (cv_.wait_until(sl, t, [this] { return stop_; }))
    break;

// Copy out shared data under a lock. Use copies outside lock.
auto const lastHeartbeat = lastHeartbeat_;
auto const armed = armed_;
sl.unlock();

// Stall detection logic ...

bool change = false;
if (app_.getJobQueue().isOverloaded())
{
    JLOG(journal_.info()) << "Raising local fee (JQ overload): "
                          << app_.getJobQueue().getJson(0);
    change = app_.getFeeTrack().raiseLocalFee();
}
else
{
    change = app_.getFeeTrack().lowerLocalFee();
}

if (change)
{
    // VFALCO TODO replace this with a Listener / observer and
    // subscribe in NetworkOPs or Application.
    app_.getOPs().reportFeeChange();
}

}

---

Runtime Evidence

Two xrpld instances were built from the same 3.3.0 source (00a178fb) on identical hardware (Intel i5-14600KF, Samsung 990 Pro NVMe), with only the LoadManager.cpp fix applied to the patched binary. Both ran as validators on XRPL mainnet simultaneously. The stock binary used production JobTypes.h thresholds throughout.

Natural Event (production thresholds, no artificial load)

During a brief consensus dispute at ledger 106292670, with 43 transactions in that ledger and production job latency thresholds in place, the monitor captured the following at the same second on both nodes:

STOCK   (unpatched): seq:106292670  load_factor:1
PATCHED (fixed):     seq:106292670  load_factor:1.25  load_factor_local:1.25

The patched node correctly detected JQ overload and raised localTxnLoadFee_. The stock node remained silent. Same hardware, same ledger, same second - the only variable was the binary.

This was also observed on a production XRPL validator (Intel Xeon Gold 6122) where load_factor_local rose to 7.45x during sustained consensus disputes - behaviour that was completely absent on unpatched nodes under identical network conditions.

Configured Event (lowered thresholds + CPU throttling)

To reliably demonstrate fee enforcement, the patched binary was rebuilt with lowered job latency thresholds in JobTypes.h (1ms avg / 5ms peak for localTransaction, transaction, and trustedProposal), and the patched node's CPU was throttled to 30% using cpulimit to sustain JQ overload. The stock binary ran unmodified at full CPU.

During a load_factor_local > 1 event on the patched node, a transaction was submitted to both nodes at the base fee of 10 drops:

STOCK   (unpatched): tx (fee=10 drops): tesSUCCESS
PATCHED (fixed):     tx (fee=10 drops): telINSUF_FEE_P

The patched node correctly rejected the 10-drop transaction with telINSUF_FEE_P. The stock node accepted it regardless of JQ state. telINSUF_FEE_P transactions are rejected at the node level before entering the ledger or consuming any network resources - which is the intended behaviour of the local fee protection mechanism.

---

Unit Tests

A new test suite LoadManager_test (5 cases, 15 assertions) covers the LoadFeeTrack mechanics that this fix restores to active use:

Test case	Description
raiseLocalFee requires two consecutive calls	Verifies the raiseCount_ < 2 hysteresis guard
lowerLocalFee decays elevated fee back to baseline	Confirms decay to kLftNormalFee (256)
lowerLocalFee at baseline returns false	Confirms noop when already at floor
lowerLocalFee resets raiseCount	Confirms raiseCount_ reset on lower
isLoadedLocal reflects fee state correctly	Confirms isLoadedLocal() tracks fee and count state

All 5 cases pass on the patched binary. On the stock binary, the code path these tests exercise is never reached during normal operation.

---

Credit

Bug identified by Nik Bougalis (@nbougalis).