Skip to content

feat: do not republish a property value whose payload is unchanged - #51

Merged
dcj merged 1 commit into
mainfrom
feat/publish-on-change-gate
Aug 13, 2026
Merged

feat: do not republish a property value whose payload is unchanged#51
dcj merged 1 commit into
mainfrom
feat/publish-on-change-gate

Conversation

@dcj

@dcj dcj commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Closes #50.

Property.publish_value() sent every value it was handed, with no comparison against what it last put on the wire. For a retained topic that is pure waste: the broker's retained store already holds that exact payload, every subscriber already has it, and the publish carries no information. At QoS 2 each one is a four-packet exchange plus a retained-store write plus fan-out.

Why the SDK owns this rather than the caller

The rounding, the datatype coercion and the empty-string encoding all happen downstream of set_value(), and they are precisely what collapses two distinct values into one payload. A caller holding a raw reading cannot know whether it will change what goes on the wire: 0.14494210481643677 and 0.14501120000000001 are genuinely different values that any caller-side change check calls "changed", and on a round_to=1 property both publish 0.1.

The mechanism

Property memoizes the last (topic, payload) pair it actually published, and skips a republish when all of: not forced, retained(), _ever_published, and the payload is byte-identical. The comparison is on the final payload, after coerced_value() and encode_empty_string().

The topic is half the key because set_node()/set_device() are public: a reparented property must not have its first publish on the new topic suppressed by the old topic's memo. It is one tuple rather than two fields so the pair cannot tear when the MQTT loop thread and an application thread publish concurrently.

Three carve-outs, each deliberate:

  • A non-retained (event) property is never gated. The broker stores nothing for it, so an identical consecutive payload is a second real event, not a redundant write. (The issue's acceptance criterion did not scope this, but its rationale is retained-specific.)
  • Retraction always publishes. set_value(None) / clear_value() must reach the broker to delete the topic.
  • Every whole-tree republish forces, via a keyword-only force threaded through refresh_tree()publish_nodes()Node.publish()publish_value(), defaulting True on the walk and False on the value path. This is load-bearing: without it a broker restarted with an empty retained store could never be repopulated, because on reconnect every payload matches what the property "last published".

get_last_published_value() was an active trap

Its body was return self.value(), behind a docstring admitting it was a placeholder. Once a real memo exists, anyone building change detection on it compares a value against itself, gets False every time, and suppresses every publish including the first. It now returns the memoized wire payload (the post-coercion, post-encoding string — an empty-string value reads back as "\x00"). Nothing in the SDK, its tests or its examples called it. This is a public-API behavior change and is in the CHANGELOG under Changed.

invalidate_publish_cache() is new, for anything that deletes a retained value topic behind the property's back: delete_all_from_mqtt() now calls it, and clear_retained_topic() documents the obligation.

Consumer impact

The retained state left on the broker is byte-for-byte identical either way: strictly fewer messages, same truth. The only consumers who notice are those inferring liveness from message arrival rather than from $state.

That is worth more than a changelog line, because this is the first producer-side suppression the SDK applies to the data plane — the four already documented in doc/consuming-a-homie-tree.md all touch only $state and $description. A consumer treating repeated identical values as a freshness heartbeat sees a settled value's topic go quiet, and quiet is indistinguishable from dead if you are counting messages. That document gains a fifth row and a paragraph saying so.

Verification

  • 581 tests pass (560 before), ruff check + ruff format --check clean on 0.16.1.

  • Mutation-tested every load-bearing line; each now fails at least one test:

    Mutation Tests failed
    add_property drops force 1
    gate ignores retained() 2
    memo key drops topic 3
    memo never written 5
    clear_value keeps stale memo 1
    Node.publish drops force hop 5
    refresh_tree gates its walk 3
    delete_all keeps stale memo 1

One note for reviewers, since the code comment says it and it is counter-intuitive: Node.add_property()'s force=True does not cover the re-add case. A fresh property passes the gate anyway, and so does one re-added after delete_property(), because clear_value() resets both gate conjuncts. What it actually covers is a value topic wiped behind the property's back, which is what test_re_announce_restores_a_value_topic_wiped_behind_the_propertys_back pins.

Known limitation, not introduced here

homie.Property has no lock. A forced republish on the MQTT loop thread interleaved with an app-thread set_value() can leave the memo one payload behind, so a later revert to that payload is suppressed until the next reconnect or genuine change. The race is pre-existing in kind (_value and _ever_published race the same way today); adding a lock would change the class's concurrency contract and hold it across an injected transport's publish(). Flagged rather than solved.

🤖 Generated with Claude Code

Property.publish_value() sent every value it was handed, with no comparison
against what it last put on the wire. A producer that recomputes and re-sets
its property set each tick therefore republished payloads the broker's
retained store already held, at QoS 2, forever.

The comparison has to live here rather than in the caller: rounding, datatype
coercion and empty-string encoding all happen downstream of set_value(), and
they are precisely what collapses two distinct values into one payload. A
caller holding a raw reading cannot know whether it will change the payload.

Property now memoizes the last (topic, payload) pair it actually published and
skips a republish when all of: not forced, retained, ever-published, and
byte-identical. The topic is half the key because set_node()/set_device() are
public, so a reparented property must not have its first publish on the new
topic suppressed by the old topic's memo. It is one tuple rather than two
fields so the pair cannot tear across threads.

Three carve-outs:

- A non-retained (event) property is never gated. The broker stores nothing
  for it, so an identical consecutive payload is a second real event.
- Retraction always publishes.
- Every whole-tree republish forces, via a keyword-only `force` threaded
  through refresh_tree() -> publish_nodes() -> Node.publish() ->
  publish_value(). Without it a broker restarted with an empty retained store
  could never be repopulated: on reconnect every payload matches what the
  property "last published", so nothing would be sent.

Also implements get_last_published_value(), whose body was `return self.value()`
behind a docstring admitting it was a placeholder. That made it an active trap
once a real memo existed: change detection built on it compares a value against
itself, is always False, and suppresses every publish including the first. It
now returns the memoized wire payload, and invalidate_publish_cache() forgets
it for anything that deletes a retained value topic behind the property's back
(delete_all_from_mqtt() now calls it; clear_retained_topic() documents it).

The retained state left on the broker is byte-for-byte identical either way:
strictly fewer messages, same truth. doc/consuming-a-homie-tree.md gains a
fifth row and a paragraph, since this is the first producer-side suppression
to touch the data plane rather than $state/$description, and a consumer
treating repeated identical values as a freshness heartbeat sees a settled
value's topic go quiet.

Closes #50.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Every value publish goes to the wire, even when the payload is byte-identical to the last one

1 participant