What happens
A successful MQTT publication can remain in pending_publishes permanently when the Paho completion callback races with putNewMessage().
putNewMessage() checks unexpected_publishes and then appends the MID to pending_publishes. If the callback runs between those two operations, it records the MID as unexpected. The publishing thread then appends the already-completed MID as pending. No later callback removes it.
Local broker proof
I reproduced this with an isolated local Mosquitto listener, a real QoS 1 publication and Paho's real network callback thread.
The fixture pauses the publishing thread after it has checked unexpected_publishes but before it appends to pending_publishes. It lets the broker acknowledgement run through the installed Sarracenia callback, then resumes the publishing thread.
The normal publication control drained both collections. The forced interleaving produced:
{
"callback_thread": ["paho-mqtt-client-"],
"control": {"pending": [], "unexpected": []},
"pending": [2],
"unexpected": [2]
}The publish itself returned success. MID 2 had already completed, but Sarracenia still owned it as pending.
Impact
MQTT.close() waits for pending_publishes to empty. This race gives it a permanently pending MID even when the broker successfully acknowledged the message, connecting this defect to the shutdown hang in #48.
Bounding close() alone does not fix the incorrect completion bookkeeping.
Expected behavior
Registration and completion reconciliation should be atomic. Whether the callback runs before, during or after registration, a completed MID should remain in neither collection.
Validation
Add deterministic before, during and after completion tests. Keep the real local broker proof for the during case, and confirm the normal control still drains. Then run the focused MQTT and broader source suites.
What happens
A successful MQTT publication can remain in
pending_publishespermanently when the Paho completion callback races withputNewMessage().putNewMessage()checksunexpected_publishesand then appends the MID topending_publishes. If the callback runs between those two operations, it records the MID as unexpected. The publishing thread then appends the already-completed MID as pending. No later callback removes it.Local broker proof
I reproduced this with an isolated local Mosquitto listener, a real QoS 1 publication and Paho's real network callback thread.
The fixture pauses the publishing thread after it has checked
unexpected_publishesbut before it appends topending_publishes. It lets the broker acknowledgement run through the installed Sarracenia callback, then resumes the publishing thread.The normal publication control drained both collections. The forced interleaving produced:
{ "callback_thread": ["paho-mqtt-client-"], "control": {"pending": [], "unexpected": []}, "pending": [2], "unexpected": [2] }The publish itself returned success. MID
2had already completed, but Sarracenia still owned it as pending.Impact
MQTT.close()waits forpending_publishesto empty. This race gives it a permanently pending MID even when the broker successfully acknowledged the message, connecting this defect to the shutdown hang in #48.Bounding
close()alone does not fix the incorrect completion bookkeeping.Expected behavior
Registration and completion reconciliation should be atomic. Whether the callback runs before, during or after registration, a completed MID should remain in neither collection.
Validation
Add deterministic before, during and after completion tests. Keep the real local broker proof for the during case, and confirm the normal control still drains. Then run the focused MQTT and broader source suites.