Skip to content

Commit 28dc569

Browse files
dcjclaude
andauthored
homie: inbound /set async dispatch, thread-safe and promoted to the root (#15) (#23)
- Dispatch: schedule an async /set callback onto the consumer's loop with asyncio.run_coroutine_threadsafe (ensure_future is not safe from paho's network thread, where /set arrives). Branch on the callback's actual return (iscoroutine), not async_loop's presence, so a SYNC callback runs inline even when a device-level loop is set. Surface exceptions the discarded scheduling Future would otherwise swallow, matching the sync path's logging. - Promotion: async_loop is now Device(async_loop=), propagated to every property via add_node()/Node.add_property() (mirroring _qos), children inheriting the root's loop. The per-Property async_loop still works and is kept when the device sets none. - Fix the Optional[...] = False default to = None (bool vs loop annotation); widen to AbstractEventLoop. - Fix Node's mutable default properties: dict = {} (shared-dict footgun the new propagation reads/writes through). docs: CHANGELOG [Unreleased] (Added Device(async_loop=), Fixed dispatch/defaults); README BYO note on async /set. Reviewed adversarially; 4 findings fixed. +10 tests. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 5f79050 commit 28dc569

4 files changed

Lines changed: 154 additions & 9 deletions

File tree

CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,14 @@ All notable changes to `ebus-sdk` are recorded here. Format follows [Keep a Chan
44

55
## [Unreleased]
66

7+
### Added
8+
9+
- `Device(async_loop=...)`: set the consumer's asyncio event loop once on the root instead of on every settable `Property`. It propagates to every property in the tree via `add_node()` / `Node.add_property()` (like QoS) and is inherited by child devices, so inbound `/set` callbacks for a tree with N settable properties no longer configure the loop N times. The per-`Property` `async_loop` argument still works and is preserved when no device-level loop is set. ([#15](https://github.com/electrification-bus/python-sdk/issues/15))
10+
11+
### Fixed
12+
13+
- Inbound `/set` on a settable property with an async (coroutine) callback is now dispatched to the consumer's event loop with `asyncio.run_coroutine_threadsafe` instead of `asyncio.ensure_future`. `/set` arrives on the transport's network-loop thread, and `ensure_future` is not safe to call from a thread other than the loop's own, so the previous path could misbehave on a real async host. The dispatch now branches on whether the callback returns a coroutine (not merely on a loop being configured), so a synchronous callback keeps running inline even when a tree-wide loop is set, and an exception raised inside an async callback is logged rather than silently swallowed by the discarded scheduling future. The `async_loop` default is corrected from `False` to `None` (a bool against the event-loop annotation, which `py.typed` surfaces), and `Node`'s mutable default `properties={}` is replaced with `None` (a shared-dict footgun the new propagation reads and writes through). ([#15](https://github.com/electrification-bus/python-sdk/issues/15))
14+
715
## [0.16.0] — 2026-08-02
816

917
### Added

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,8 @@ client.connect() # host connects on its own loop
7474

7575
`device.will()` returns the tree's Last Will descriptor and `device.refresh_tree()` republishes the whole tree; the `set_will` / `on_connect` / `connect` calls above are illustrative of your host's own MQTT API. Property values publish once the client is connected (the SDK gates on `is_connected()`, not on its own `start()`, which a caller-driven client never calls). `device.stop()` publishes a final retained `$state=disconnected` through the client and returns immediately, without flushing or closing it. `on_disconnect=` is inert for an injected client; register disconnect handling on your own client.
7676

77+
For the inbound direction, if the tree has settable properties whose callbacks are async coroutines, pass `Device(async_loop=<your event loop>)`: inbound `/set` arrives on the transport's network thread, and this schedules the callback onto your loop (set once for the whole tree, not per property). A synchronous callback runs inline and needs no loop.
78+
7779
**Two host shapes, and the will's limit.** The example above is a caller that owns a *dedicated* client and connects it itself, so it can set the will before connect. A host with a *shared* connection it does not own (Home Assistant is the archetype: one connection, up before your code loads, its will already set from the host's own config with no hook to change it) cannot set an eBus will, because MQTT allows one will per connection. There, wire `refresh_tree()` to the host's reconnect callback and let the SDK gate on `is_connected()`: the tree stays correct across every reconnect the process survives. (A consequence to be aware of: on such a shared connection the SDK cannot set the will's `$state=lost`, since the will is a CONNECT-time property the host owns.)
7880

7981
#### Clearing a value vs. an empty-string value

src/ebus_sdk/homie.py

Lines changed: 45 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -478,7 +478,7 @@ def __init__(
478478
supports_target: Optional[bool] = False,
479479
node: Optional[Node] = None,
480480
device: Optional[Device] = None,
481-
async_loop: Optional[asyncio.SelectorEventLoop] = False,
481+
async_loop: Optional[asyncio.AbstractEventLoop] = None,
482482
from_dict: Optional[dict] = None,
483483
):
484484
if from_dict:
@@ -973,13 +973,35 @@ def _settable_callback(self, topic: str, payload: Union[bytes, bytearray]) -> No
973973
# Property supports_target, publish that!
974974
self.publish_target_value(payload)
975975
# Call the property's set_callback function
976-
if self.async_loop:
977-
asyncio.ensure_future(set_callback(payload), loop=self.async_loop)
978-
else:
979-
set_callback(payload)
976+
# Run the callback: a sync callback runs inline here (on the transport's
977+
# network thread, as before); an async (coroutine) callback is scheduled onto
978+
# the consumer's event loop thread-safely via run_coroutine_threadsafe
979+
# (ensure_future is NOT safe to call from a thread other than the loop's own).
980+
# Decide on the callback's actual return, not just async_loop's presence, so a
981+
# sync callback stays inline even when a device-level loop is set for the tree.
982+
result = set_callback(payload)
983+
if self.async_loop is not None and asyncio.iscoroutine(result):
984+
future = asyncio.run_coroutine_threadsafe(result, self.async_loop)
985+
# The Future is otherwise discarded, so an exception raised inside the
986+
# coroutine would vanish silently (the inline path is caught below by the
987+
# surrounding try/except). Surface it, matching the sync path's logging.
988+
future.add_done_callback(partial(self._log_async_set_result, property_id=property_id))
980989
except Exception as e:
981990
logger.exception(f"reason=propertySetCallbackException,e={e}")
982991

992+
def _log_async_set_result(self, future, property_id) -> None:
993+
"""Done-callback for an async /set handler scheduled via run_coroutine_threadsafe.
994+
995+
The scheduling Future is otherwise discarded, and a discarded concurrent.futures
996+
Future swallows a stored exception silently (unlike an asyncio.Task). Surface it,
997+
matching the synchronous path's ``propertySetCallbackException`` logging.
998+
"""
999+
if future.cancelled():
1000+
return
1001+
exc = future.exception()
1002+
if exc is not None:
1003+
logger.error(f"reason=propertySetAsyncCallbackException,propertyID={property_id},exc={exc!r}")
1004+
9831005
def set_subscribe(self) -> None:
9841006
"""
9851007
Subscribe to property/set topic on Homie broker
@@ -1022,7 +1044,7 @@ def __init__(
10221044
id: Optional[str] = None,
10231045
name: Optional[str] = None,
10241046
type: Optional[str] = None,
1025-
properties: dict = {},
1047+
properties: Optional[dict] = None,
10261048
device: Optional[Device] = None,
10271049
# mqttc: Optiona[MqttClient] = None, # DCJ pretty sure we can remove this
10281050
from_dict: Optional[dict] = None,
@@ -1047,7 +1069,7 @@ def __init__(
10471069
else:
10481070
self._name = id
10491071
self._type = type
1050-
self._properties = properties
1072+
self._properties = properties if properties is not None else {}
10511073
self._device = device
10521074

10531075
def as_dict(self) -> dict:
@@ -1112,9 +1134,11 @@ def add_property(self, property: Property) -> Property:
11121134
"""
11131135
if not property.node():
11141136
property.set_node(self)
1115-
# Propagate QoS from device if available
1137+
# Propagate QoS (and the async /set dispatch loop, if set) from the device.
11161138
if self._device and hasattr(self._device, "_qos"):
11171139
property._qos = self._device._qos
1140+
if self._device and getattr(self._device, "_async_loop", None) is not None:
1141+
property.async_loop = self._device._async_loop
11181142
# Note set_subscribe() checks if property is settable...
11191143
property.set_subscribe()
11201144
# Add property to dictionary BEFORE publishing description
@@ -1331,6 +1355,7 @@ def __init__(
13311355
mqtt_cfg: Optional[dict] = None,
13321356
mqttc: Optional[MqttDeviceTransport] = None,
13331357
qos: int = EBUS_HOMIE_MQTT_QOS,
1358+
async_loop: Optional[asyncio.AbstractEventLoop] = None,
13341359
on_disconnect: Optional[Callable[[bool], None]] = None,
13351360
):
13361361
# Root vs. child invariants — mutually exclusive. Test presence by identity
@@ -1373,6 +1398,14 @@ def __init__(
13731398
self._owned_client: Optional[MqttClient] = None
13741399
self._state = None
13751400
self._qos = qos
1401+
# The consumer's asyncio event loop for dispatching inbound /set callbacks on
1402+
# settable properties. Set once per tree on the root and propagated to every
1403+
# property via add_node() / Node.add_property() (like _qos), rather than
1404+
# per-property. Children inherit the root's loop unless given their own. None
1405+
# means /set callbacks run synchronously on the transport's network thread.
1406+
self._async_loop = (
1407+
async_loop if async_loop is not None else (parent._async_loop if parent is not None else None)
1408+
)
13761409
# Optional consumer hook: called on the ROOT's MQTT (dis)connect. Only a
13771410
# root owns a client (children share it), so it fires on the root only.
13781411
# Contract is transport-neutral: on_disconnect(clean: bool), never a paho
@@ -1702,9 +1735,12 @@ def add_node(self, node: Node) -> Node:
17021735
"""
17031736
if not node.device():
17041737
node.set_device(self)
1705-
# Propagate device QoS to all properties in this node
1738+
# Propagate device QoS (and the async /set dispatch loop, if set) to every
1739+
# property in this node.
17061740
for prop in node.properties().values():
17071741
prop._qos = self._qos
1742+
if self._async_loop is not None:
1743+
prop.async_loop = self._async_loop
17081744
node_id = node.id()
17091745
self._nodes.update({node_id: node})
17101746
node.publish()

tests/test_homie_device.py

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2029,6 +2029,105 @@ def test_owned_client_handle_is_set_and_cleared_for_an_sdk_built_client(self):
20292029
assert device._owned_client is None
20302030

20312031

2032+
class TestInboundAsyncLoop:
2033+
"""Inbound /set async dispatch: async_loop promoted to the root, thread-safe (#15)."""
2034+
2035+
def test_async_loop_default_is_none(self):
2036+
# Was `Optional[...] = False` (a bool against the loop annotation); now None.
2037+
assert Property(id="mode").async_loop is None
2038+
2039+
def test_device_async_loop_propagates_via_add_node(self):
2040+
loop = MagicMock()
2041+
prop = Property(id="setpoint", datatype=PropertyDatatype.FLOAT)
2042+
node = Node(id="core", name="Core", type="sensor", properties={"setpoint": prop})
2043+
Device(id="dev", async_loop=loop, nodes=[node]) # add_node propagates to the node's props
2044+
assert prop.async_loop is loop
2045+
2046+
def test_device_async_loop_propagates_via_node_add_property(self):
2047+
loop = MagicMock()
2048+
device = Device(id="dev", async_loop=loop) # transport-free root
2049+
node = Node(id="core", name="Core", type="sensor")
2050+
device.add_node(node)
2051+
prop = Property(id="setpoint", value=1.0, datatype=PropertyDatatype.FLOAT)
2052+
node.add_property(prop) # node is attached -> propagates the device's loop
2053+
assert prop.async_loop is loop
2054+
2055+
def test_child_device_inherits_root_async_loop(self):
2056+
loop = MagicMock()
2057+
root = Device(id="root", async_loop=loop) # transport-free root
2058+
child = Device(id="child", parent=root)
2059+
assert child._async_loop is loop # one loop per tree, inherited by children
2060+
node = Node(id="core", name="Core", type="sensor")
2061+
child.add_node(node)
2062+
prop = Property(id="setpoint", value=1.0, datatype=PropertyDatatype.FLOAT)
2063+
node.add_property(prop)
2064+
assert prop.async_loop is loop
2065+
2066+
def test_async_dispatch_uses_run_coroutine_threadsafe(self):
2067+
"""An async /set callback is scheduled onto the consumer's loop thread-safely;
2068+
/set arrives on the transport's network thread, so ensure_future is unsafe."""
2069+
2070+
async def cb(payload):
2071+
return None
2072+
2073+
loop = MagicMock()
2074+
prop = Property(id="mode", settable=True, set_callback=cb)
2075+
prop.async_loop = loop
2076+
with patch("asyncio.run_coroutine_threadsafe") as rct, patch("asyncio.ensure_future") as ef:
2077+
prop._settable_callback("ebus/5/dev/node/mode/set", b"LOAD_UP")
2078+
rct.assert_called_once()
2079+
assert rct.call_args.args[1] is loop # scheduled onto the given loop
2080+
ef.assert_not_called() # never the thread-unsafe ensure_future path
2081+
rct.call_args.args[0].close() # close the un-awaited coroutine (rct is mocked)
2082+
2083+
def test_sync_dispatch_when_no_loop(self):
2084+
seen = []
2085+
prop = Property(id="mode", settable=True, set_callback=lambda v: seen.append(v))
2086+
assert prop.async_loop is None
2087+
with patch("asyncio.run_coroutine_threadsafe") as rct:
2088+
prop._settable_callback("ebus/5/dev/node/mode/set", b"LOAD_UP")
2089+
assert seen == ["LOAD_UP"] # invoked synchronously
2090+
rct.assert_not_called()
2091+
2092+
def test_sync_callback_stays_inline_under_device_loop(self):
2093+
"""A device-level loop must not force a sync callback onto the async path: the
2094+
dispatch branches on the callback's return, not on the loop's presence (#15)."""
2095+
loop = MagicMock()
2096+
seen = []
2097+
prop = Property(id="mode", settable=True, set_callback=lambda v: seen.append(v))
2098+
prop.async_loop = loop # a device-level loop is propagated to every property
2099+
with patch("asyncio.run_coroutine_threadsafe") as rct:
2100+
prop._settable_callback("ebus/5/dev/node/mode/set", b"ON")
2101+
assert seen == ["ON"] # ran inline, once
2102+
rct.assert_not_called() # not scheduled: a sync callback returns no coroutine
2103+
2104+
def test_per_property_loop_survives_when_device_has_no_loop(self):
2105+
"""Backward compat: a per-Property async_loop is kept when the device sets none."""
2106+
loop_a = MagicMock()
2107+
prop = Property(id="setpoint", datatype=PropertyDatatype.FLOAT, async_loop=loop_a)
2108+
node = Node(id="core", name="Core", type="sensor", properties={"setpoint": prop})
2109+
Device(id="dev", nodes=[node]) # no async_loop -> propagation is skipped
2110+
assert prop.async_loop is loop_a
2111+
2112+
def test_device_loop_overrides_per_property_loop(self):
2113+
"""One loop per tree: a device-level loop wins over a pre-set per-Property loop."""
2114+
loop_a, loop_b = MagicMock(), MagicMock()
2115+
prop = Property(id="setpoint", datatype=PropertyDatatype.FLOAT, async_loop=loop_a)
2116+
node = Node(id="core", name="Core", type="sensor", properties={"setpoint": prop})
2117+
Device(id="dev", async_loop=loop_b, nodes=[node])
2118+
assert prop.async_loop is loop_b
2119+
2120+
def test_async_set_callback_exception_is_surfaced(self):
2121+
"""A raising async /set handler is logged, not swallowed by the discarded Future."""
2122+
prop = Property(id="mode")
2123+
future = MagicMock()
2124+
future.cancelled.return_value = False
2125+
future.exception.return_value = ValueError("bad setpoint")
2126+
with patch("ebus_sdk.homie.logger") as mock_logger:
2127+
prop._log_async_set_result(future, "mode")
2128+
assert any("propertySetAsyncCallbackException" in str(c) for c in mock_logger.error.call_args_list)
2129+
2130+
20322131
class TestDeviceWithoutTransport:
20332132
"""`mqtt_cfg=None` — the declared default — builds a device tree with no transport."""
20342133

0 commit comments

Comments
 (0)