feat: declare_lost() and stop(announce=False) (#52)

Device modeled three teardowns and implemented one. A producer that knew it was
failing could announce disconnected (a lie) or reach around the SDK to its client.

declare_lost() is tree-level like will() and stop(): it publishes the ROOT $state,
which per the Homie 5 effective-state rule covers every descendant, and emits exactly
what will() describes so the declared and will-driven paths cannot drift. The state
move and the publish happen together and the move is unconditional, because publishing
a state the Device does not hold is how a later refresh_tree() republishes ready over
it. Returns whether $state moved, per set_state convention.

stop(announce=False) tears down without publishing. The state move now lives inside
the announcing branch so it cannot overwrite a just-declared lost.

Neither addition widens the injected-transport surface.

Reported by @cayossarian. Closes #46.

Author: dcj

CHANGELOG.md

@@ -6,6 +6,10 @@ All notable changes to `ebus-sdk` are recorded here. Format follows [Keep a Chan
 
 ### Added
 
+- `Device.declare_lost()`: a way to announce deliberate death. `Device` modeled three teardowns and implemented one, so a producer that knew it was failing (a fatal error handler, a supervisor about to kill it, hardware that has gone away, a simulator acting the part) could only announce `disconnected`, which is a lie, or reach around the SDK to the concrete client; `DeviceState.LOST` was published nowhere in `homie.py` except inside the `will()` descriptor, and the will fires only on an *unclean* disconnect, which the clean disconnect `stop()` performs deliberately suppresses. It is TREE-level like `will()` and `stop()`, publishing the ROOT's `$state` (per the Homie 5 effective-state rule that covers every descendant in one publish) and emitting exactly the topic and payload `will()` describes, so the declared and will-driven paths cannot drift; to mark a single device lost, `set_state(DeviceState.LOST)` on that device remains the right call, and `declare_lost()` would blank the whole tree's liveness. The state move and the publish happen together, and the move is unconditional, because publishing a state the `Device` does not hold is exactly how a later `refresh_tree()` silently republishes `ready` over it. It returns whether `$state` actually moved, reusing `set_state`'s True-changed/False-already-there convention: on an injected transport that distinguishes "queued, now drain" from "already lost, nothing to wait for", and it is deliberately not a delivery signal, which it could not honestly be there. Owned clients flush; injected clients queue on the caller's loop, since `publish_and_flush` is owned-only and off the `MqttDeviceTransport` surface. Reported by [@cayossarian](https://github.com/cayossarian), whose async-transport drain sequence shaped the contract, and adopted by `ebus-panel-sim` in place of the reach-around that produced four separate downstream bugs. ([#46](https://github.com/electrification-bus/python-sdk/issues/46))
+
+- `Device.stop(announce=False)`: tear down without publishing anything, leaving the retained `$state` exactly as it stands. The counterpart to `declare_lost()`: the default `announce=True` would overwrite a just-declared `lost` with `disconnected`, and the state move now lives inside the announcing branch so it cannot. Named `announce` rather than the `graceful` a downstream reached for, because "graceful" conflates the announcement with the bounded clean disconnect, and the teardown stays bounded and clean in both modes; only the announcement differs. Unpaired it leaves whatever was published last, typically a stale `ready`, and nothing will correct that, so the docstring and the README say so plainly. Adds nothing to the injected-transport surface: it only skips a publish, never adds a call. ([#46](https://github.com/electrification-bus/python-sdk/issues/46))
+
 - `Property.invalidate_publish_cache()`: forget what a property last published, for anything that deletes its retained value topic behind its back. The publish-on-change skip below assumes the broker still holds the payload the property last sent, so an operator wiping the broker, or a call to `Device.clear_retained_topic()` aimed at a value topic, leaves that assumption false and the next `set_value()` of the same value would be skipped against an empty topic. `Device.delete_all_from_mqtt()` now calls it on every property it clears; `clear_value()` and `Node.delete_property()` reset the memo themselves, so only the raw-topic paths need it. Distinct from `_ever_published`: this says "I no longer know what the broker holds", not "I have never published". ([#50](https://github.com/electrification-bus/python-sdk/issues/50))
 
 ### Changed

README.md

@@ -72,7 +72,7 @@ client.on_connect(device.refresh_tree)         # re-announce the retained tree o
 client.connect()                               # host connects on its own loop
 ```
 
-`device.will()` returns the tree's Last Will descriptor and `device.refresh_tree()` republishes the whole tree; the `set_will` / `on_connect` / `connect` calls above are illustrative of your host's own MQTT API. Property values publish once the client is connected (the SDK gates on `is_connected()`, not on its own `start()`, which a caller-driven client never calls). `device.stop()` publishes a final retained `$state=disconnected` through the client and returns immediately, without flushing or closing it. `on_disconnect=` is inert for an injected client; register disconnect handling on your own client.
+`device.will()` returns the tree's Last Will descriptor and `device.refresh_tree()` republishes the whole tree; the `set_will` / `on_connect` / `connect` calls above are illustrative of your host's own MQTT API. Property values publish once the client is connected (the SDK gates on `is_connected()`, not on its own `start()`, which a caller-driven client never calls). `device.stop()` publishes a final retained `$state=disconnected` through the client and returns immediately, without flushing or closing it; `device.stop(announce=False)` publishes nothing and leaves the retained `$state` as it stands, for a caller that published its own final state first (see `declare_lost()` below). `on_disconnect=` is inert for an injected client; register disconnect handling on your own client.
 
 For the inbound direction, if the tree has settable properties whose callbacks are async coroutines, pass `Device(async_loop=<your event loop>)`: inbound `/set` arrives on the transport's network thread, and this schedules the callback onto your loop (set once for the whole tree, not per property). A synchronous callback runs inline and needs no loop.
 
@@ -95,10 +95,33 @@ That matters because ordering is a guarantee the SDK maintains on your behalf. A
 
 So if your transport hands work onward rather than publishing inline (an `MqttDeviceTransport` over a natively-async client, say, where `publish()` enqueues and returns), **drain a single queue in order** rather than dispatching each publish independently. The failure is invisible in testing: it holds by luck under a fast broker and breaks under a slow first publish.
 
-`publish()` returning without having reached the wire is otherwise entirely legitimate: the protocol types every return as `object` precisely because the SDK discards them. It does mean teardown needs a drain point of your own, because `Device.stop()` publishes the final `$state` and returns without flushing. Close the queue before closing the client, or that last message is lost behind it.
+`publish()` returning without having reached the wire is otherwise entirely legitimate: the protocol types every return as `object` precisely because the SDK discards them. It does mean teardown needs a drain point of your own, because `Device.stop()` publishes the final `$state` and returns without flushing. Close the queue before closing the client, or that last message is lost behind it. The same obligation applies to `declare_lost()`, which queues the `lost` and returns; its `True` return means "queued, now drain", `False` means "already lost, nothing to wait for".
 
 **A producer should own its MQTT connection.** The example above owns a *dedicated* client and connects it itself, so the SDK's Last Will (`$state=lost` on an ungraceful death) works normally: prefer this for any producer whose liveness matters. A *shared* connection owned by a host (Home Assistant is the archetype: one connection, up before your code loads, its single will already spent on the host's own) **cannot carry an eBus will**, because MQTT allows one will per connection. A producer publishing through such a connection therefore never signals ungraceful death: a crash leaves a stale retained `$state=ready`, and consumers render a dead device as alive. Reconnect is still handled (wire `refresh_tree()` to the host's reconnect callback and gate on `is_connected()`), but permanent death is not, and there is no portable substitute (a host that owns the connection also will not forward the MQTT 5 publish properties that would let `$state` expire). So do not publish a liveness-bearing device through a connection you do not own: if a host environment forbids a dedicated connection, run the producer as a **separate adapter** with its own connection rather than borrowing the host's. (The injected-client seam is still the right tool for the *consumer* role, `Controller(mqttc=...)`, which has no `$state` and no will to lose, and for tests.)
 
+#### Announcing death: the three teardowns
+
+A device has three ways to stop, and they mean different things to a consumer rendering `$state` into availability:
+
+| Teardown | `$state` left retained | How |
+| --- | --- | --- |
+| Graceful shutdown | `disconnected` | `device.stop()` |
+| Ungraceful death (crash, power loss) | `lost` | the Last Will, which fires only on an *unclean* disconnect |
+| Deliberate death | `lost` | `device.declare_lost()` |
+
+The third is for a producer that knows it is failing: a fatal error handler, a supervisor about to kill it, hardware that has gone away, or a simulator acting the part. `disconnected` would tell consumers the shutdown was orderly and expected, which is a lie.
+
+```python
+device.declare_lost()          # root's $state=lost, published and (owned path) flushed
+device.stop(announce=False)    # tear down without overwriting it with `disconnected`
+```
+
+`declare_lost()` is **tree-level**, like `will()` and `stop()`: it publishes the *root's* `$state`, which per the Homie 5 effective-state rule makes every descendant lost too, and it publishes exactly the topic and payload `will()` describes so the two paths cannot drift. To mark one device lost (a proxy whose single upstream vanished), use `set_state(DeviceState.LOST)` on that device instead.
+
+It moves the state and publishes it together, and the move is unconditional: publishing a state the `Device` does not hold is how a later `refresh_tree()` silently republishes `ready` over it. It returns whether `$state` actually moved, the same convention as `set_state`: on an injected transport, a `True` from a connected tree is your cue to drain, and `False` means the root was already lost. It is not a delivery signal and cannot be one there. Publishing is skipped entirely when the broker is unreachable (the state still moves, and the next connect republishes it), so `True` does not by itself prove anything was queued. It does not stop the client.
+
+`stop(announce=False)` unpaired leaves whatever was published last, typically a stale `ready`, and nothing will correct it: the clean disconnect `stop()` performs suppresses the LWT. Neither call substitutes for the will, because a crashed process calls nothing.
+
 #### Clearing a value vs. an empty-string value
 
 Homie 5 distinguishes two things that both look "empty" on the wire, and the SDK handles each automatically:
@@ -147,7 +170,7 @@ Device(id='mid-1', type='...metering', parent=bess)
 panel.children()[0].delete()
 ```
 
-Children may have children of their own. A single Last Will registered on the root marks the entire tree `lost` if the publisher process dies — controllers compute effective state per the Homie 5 precedence table (see [`HOMIE_EFFECTIVE_STATE_TABLE`](src/ebus_sdk/homie.py)).
+Children may have children of their own. A single Last Will registered on the root marks the entire tree `lost` if the publisher process dies, and `root.declare_lost()` publishes the same thing deliberately when the publisher knows it is dying — controllers compute effective state per the Homie 5 precedence table (see [`HOMIE_EFFECTIVE_STATE_TABLE`](src/ebus_sdk/homie.py)).
 
 `$description` republishes are minimized: structural changes made inside one `state_transition()` collapse to a single consolidated publish at exit (not one per `add_node`), and `publish_description()` is a no-op when the description content (ignoring its `version` timestamp) is unchanged — so a `state_transition()` that changes nothing structural does not re-emit the (potentially multi-KB) `$description`. A reconnect always republishes regardless, to restore retained state. Note this suppresses the redundant `$description` payload, not the `$state` `init`→`ready` edge of an empty transition. Property *values* are minimized the same way and with the same reconnect carve-out (see [Unchanged values are not republished](#unchanged-values-are-not-republished)).
 
@@ -256,7 +279,7 @@ MQTT transport lives in the separate [`ebus-mqtt-client`](https://github.com/ele
 
 Core Homie convention implementation:
 
-- **Device** - Represents a Homie device; pass `parent=` to build a child in a tree, or `on_disconnect=` for a push disconnect hook (`clean: bool`)
+- **Device** - Represents a Homie device; pass `parent=` to build a child in a tree, or `on_disconnect=` for a push disconnect hook (`clean: bool`); `declare_lost()` announces deliberate death and `stop(announce=False)` tears down without announcing
 - **Node** - Groups related properties within a device
 - **Property** - Individual data points (sensors, controls)
 - **Controller** - Discovers and monitors Homie devices on a broker; navigates trees and computes effective state; `set_on_disconnect_callback` for push disconnect notification

doc/building-a-proxy.md

@@ -147,13 +147,13 @@ A proxy is not one flat device. Per the eBus [`proxy.md`](https://github.com/ele
 - Publish **one child device per proxied device**, each `Device(id=..., type=..., parent=root)`. The proxied measurements live here.
 - Name each child `{proxier-id}-{proxied-id}` (the proxied id is the device's stable serial when it has one). Consumers correlate a proxy and a native publisher of the same physical device by `info/serial-number`, not by device id.
 
-Children share the root's single MQTT connection automatically (that is what `parent=` does), and one Last Will on the root marks the whole tree `lost` if the process dies. See [Device Trees](../README.md#device-trees-parent--child) in the README.
+Children share the root's single MQTT connection automatically (that is what `parent=` does), and one Last Will on the root marks the whole tree `lost` if the process dies; `root.declare_lost()` publishes exactly the same thing deliberately, when the bridge knows it is dying rather than crashing. See [Device Trees](../README.md#device-trees-parent--child) in the README.
 
 ## Lifecycle and state
 
 - **Batch structural changes.** Adding N nodes/properties inside one `with device.state_transition():` collapses to a single `$description` publish and one `init` to `ready` edge, instead of N. Always build a device's structure inside a transition.
 - **Connect before you publish.** `Device(..., mqtt_cfg=...)` connects asynchronously. If you build and publish before the broker connection is established, the first retained `$description` / `$state` the broker keeps can be a pre-connect snapshot until the SDK's on-connect refresh corrects it. Wait for `device.mqttc.is_connected()` before the initial build so the first retained state is correct.
-- **Drive `$state` from availability.** When your upstream reports a device offline, set the child `DeviceState.LOST` (and `READY` when it returns). The root's Last Will covers process death.
+- **Drive `$state` from availability.** When your upstream reports ONE device offline, `set_state(DeviceState.LOST)` on that child (and `READY` when it returns). When the whole bridge is dying, `root.declare_lost()` publishes the root's `$state=lost`, which per the Homie 5 effective-state rule covers every descendant in a single publish; follow it with `stop(announce=False)` so the teardown does not overwrite it with `disconnected`. Do not reach for `declare_lost()` for one dead upstream: it blanks the entire tree's liveness. The root's Last Will still covers process death, which neither call can, since a crashed process calls nothing.
 
 ## Settable / bidirectional properties (control back to the device)
 

doc/consuming-a-homie-tree.md

@@ -117,7 +117,7 @@ Read that re-arming as a warning. If you take the first call as a barrier and st
 
 Two things `is_tree_complete()` deliberately does not mean:
 
-- **Not liveness.** A device counts as described once its `$description` has been parsed, whatever its `$state`. A declared child that is `lost` has still told you what it is. Use `get_effective_state()` for liveness.
+- **Not liveness.** A device counts as described once its `$description` has been parsed, whatever its `$state`. A declared child that is `lost` has still told you what it is. Use `get_effective_state()` for liveness. Note `lost` is not always a crash: a producer that knows it is dying can publish it deliberately (`Device.declare_lost()`), so you may see it arrive from a publisher that is otherwise healthy and still connected. Your obligation is unchanged, which is the point: react to the state you are told, never to how you imagine it was produced.
 - **Not permanence.** It is true of the tree you can see right now. It says nothing about the tree a second from now.
 
 ## Checklist

doc/ha-discovery-bridge.md

@@ -140,9 +140,9 @@ Home Assistant's registry and recorder internals evolve between releases; the `(
 
 - Device discovered, or `$description` changed: publish (retained) `homeassistant/device/<id>/config`.
 - Device removed (empty retained `$state`): publish an empty retained payload to that config topic, so HA drops the device.
-- Transient offline (`lost` / `disconnected`): handled by the availability template. The entity shows unavailable but is NOT removed. Set `clear_on_lost=True` if you would rather clear discovery on `lost` as well.
+- Transient offline (`lost` / `disconnected`): handled by the availability template. The entity shows unavailable but is NOT removed. Set `clear_on_lost=True` if you would rather clear discovery on `lost` as well. Note `lost` may be a deliberate declaration (`Device.declare_lost()`) rather than a crash, so a `clear_on_lost=True` deployment drops the entity's discovery config on an intentional retirement too. That is usually what you want for a retirement and rarely what you want for a transient upstream fault; drive transient faults from the affected child's own `$state` instead.
 - `bridge.stop()` is a graceful shutdown: it restores the Controller's prior callbacks but LEAVES the discovery configs it published, so Home Assistant keeps the exported entities across a bridge restart (they read values directly from the `ebus/` topics and keep working while the bridge is down). The bridge is a context manager, so a `with HaDiscoveryBridge(controller) as bridge:` block guarantees `stop()` runs on exit.
-- `bridge.clear_all()` is permanent retirement: it removes every discovery config the bridge published (empty retained payload), so Home Assistant drops those devices. Call it when the bridge and its exported devices are going away for good; `clear_on_stop=True` routes `stop()` through it. This mirrors the device lifecycle exactly: `stop()` is to the bridge what a graceful device shutdown (leave `$state` retained) is to a device, and `clear_all()` is what `Device.delete()` is.
+- `bridge.clear_all()` is permanent retirement: it removes every discovery config the bridge published (empty retained payload), so Home Assistant drops those devices. Call it when the bridge and its exported devices are going away for good; `clear_on_stop=True` routes `stop()` through it. This mirrors the device lifecycle exactly: `stop()` is to the bridge what `Device.stop()` (announce `disconnected`, leave the retained data) is to a device, and `clear_all()` is what `Device.delete()` is.
 
 `HaDiscoveryBridge` chains onto (does not clobber) any callbacks already registered on the Controller: its handler runs first, then the pre-existing one.