Fix device merge, position wipe, staleness, home_id gate; add gateway-driven home key extraction

[Cloudore]

Summary

Four bug fixes around the BLE coordinator and a new config-flow path that pulls the AES home key out of a PowerView Gen3 gateway over HTTP and persists it in .storage/, so users no longer have to hand-edit const.py (and have it wiped on every HACS update).

All on top of 37a4907. Verified against a live 10-shade installation with a mix of encrypted KDT curtains and unencrypted ACR rollers.

The bugs and the fixes

1. Device registry merge from a non-unique identifier (coordinator.py)

device_info was returning (DOMAIN, self.name) as one of the registry identifiers. self.name is whatever the BLE adapter currently reports as the local name — which, on at least one batch of shades in the wild, transiently switches to the string "AWEI T13 Pro" (a generic earbud firmware fallback, presumably picked up via a stack collision or factory misconfiguration). When two shades pass that string through device_info on the same coordinator pass, HA's device registry merges them into a single device record because they share an identifier. The user's name_by_user, area_id, and HomeKit exposure all go with it.

I hit this in my own install with four KDT curtains collapsed into one "AWEI T13 Pro" device.

Fix: identifier is now (DOMAIN, format_mac(self.address)). MACs are immutable; local names are not.

If anyone else is already living with merged records, the patch alone won't un-merge them — HA's registry doesn't reverse the operation automatically. A one-off .storage/core.device_registry edit (split the merged record into N records, reassign entities by the MAC encoded in their unique_id) was sufficient in my case. Happy to ship that as a separate utility if you'd like.

2. Position is wiped on every BLE event (coordinator.py)

self.data = {ATTR_RSSI: service_info.rssi}
if change == bluetooth.BluetoothChange.ADVERTISEMENT:
    self.data.update(self.api.dec_manufacturer_data(...))

On any event where dec_manufacturer_data returns [] (advert without a V2 payload — happens during connect, scan responses, some BLE proxies), self.data ends up with only rssi. Then:

• current_cover_position resolves to None
• is_closed returns None == 0 ⇒ False
• is_opening/is_closing fall through their guards
• CoverEntity.state settles on STATE_OPEN

So a closed shade flickers to state="open", current_position=None every time a non-V2 packet arrives. The cover entity then writes that to state history, and HomeKit / Lovelace / dashboards show the wrong thing.

Fix: preserve the last-known V2 fields across events. Only refresh ATTR_RSSI unconditionally; only update positional fields when a V2 frame actually decoded. State-of-the-moment flags (is_opening, is_closing, battery_charging, etc.) are still cleared per-event so they don't latch after movement ends.

3. Stale entities masquerade as fresh (coordinator.py, cover.py, sensor.py, binary_sensor.py)

PassiveBluetoothCoordinatorEntity.available treats any BLE address activity as "device present", so a shade that's been out of range for days keeps reporting its last-known position as gospel. I had a Baby Room Curtains motor showing state="closed" for three days after the BLE proxy lost it.

Fix: track _last_v2_ts on the coordinator. New data_available property is True iff a V2 advert decoded within STALE_AFTER = 300s. Cover/sensor/binary_sensor override available to AND with it. RSSI sensor is exempt because it tracks raw link activity.

4. home_id stickiness disables UI controls (coordinator.py + cover.py)

This one's a knock-on from fix #2 — surfaced rather than introduced.

cover.supported_features returns 0 when data["home_id"] is truthy and len(HOME_KEY) != 16. The intent is sensible: "we know encryption is needed but we don't have a key, so don't pretend the slider works." The old behaviour accidentally bypassed this because home_id was wiped every event — most of the time the gate didn't fire.

With self.data preserved, the gate now fires consistently, including on ACR roller shades that report home_id != 0 but in practice accept plaintext commands just fine (verified against my five ACR rollers). So the gate disables real-world-working controls.

Fix in this PR: home_id is consumed inline (drives api.encrypted) and popped from self.data so the gate doesn't see it. UI controls stay visible; the cipher is still applied when both home_id and a real 16-byte HOME_KEY are present.

Alternative I considered: drop the home_id check from supported_features entirely. Both approaches reach the same place. Happy to swap if you prefer the explicit removal.

The new feature: gateway-extracted home key

The HOME_KEY problem in const.py is the biggest practical pain. HACS overwrites it on every update — so users who took the time to extract their key by running scripts/extract_gateway3_homekey.py get silently broken once a month.

This PR moves the key to durable per-installation storage, and adds an in-app flow to extract it:

• gateway.py — async aiohttp port of scripts/extract_gateway3_homekey.py. Probes a Gen3 gateway, walks the shade list, and asks each in turn for the home key. Any single successful GetShadeKey response is sufficient (all shades in one home share the key). Tolerates per-shade failures (offline, out of range from the gateway, etc.).

• key_store.py — thin wrapper over homeassistant.helpers.storage.Store that persists the 16-byte key as hex in .storage/hunterdouglas_powerview_ble. Outside the integration source tree, so HACS updates leave it alone. The Store versioning is in place for future schema changes.

• config_flow.py — top-level menu now offers:

  • Extract home key from a PowerView gateway — one-shot helper. Manual URL input, or auto-prefilled from zeroconf discovery. After a successful extraction, the flow aborts with home_key_saved, reloads every existing shade entry so they pick up the new key, and never creates an entry of its own. The gateway is only needed for that one call; after that it can be unplugged, taken offline, anything.
  • Pair a PowerView shade (Bluetooth) — your existing flow, unchanged.

• Zeroconf discovery — manifest adds _PowerView-G3._tcp.local. and _powerview-g3._tcp.local. (Gen3 gateways advertise the mixed-case form in the wild; the lowercase variant is defensive). When a gateway appears on the network and no key is stored yet, the extraction flow is offered via Discovered tile. After a key is stored, the discovery handler aborts silently with home_key_already_saved to avoid spamming the UI on every rediscovery — re-extraction is always available from the integration menu.

• coordinator.py — now accepts home_key as a constructor parameter (resolved from the store at setup time in __init__.py). const.HOME_KEY remains as a fallback (default b"") for users who haven't run the extractor yet; it still works as before if you put the key there manually.

The result: the user runs the extractor once, the key sits in .storage/, every shade picks it up at coordinator init, and HACS updates stop breaking encrypted shades.

Opinionated bits worth your review

These three I'd appreciate a call from you on:

1. integration_type: "device" → "hub" in manifest. Strictly speaking each shade is still its own config entry, so "device" is defensible. I bumped it because the menu UX (manual key extraction) reads more like a hub-level operation, and "hub" makes the integration easier to discover in "Add Integration" without an existing BLE discovery. Easy to revert if you prefer.

2. home_id pop workaround in coordinator.py vs. removing the gate from supported_features. Both restore controls; the pop is less invasive to the cover class but leaves a hidden side effect. Your call.

3. Version bump to 0.24 — bumped because the config flow shape changed. You may want to use a different number or hold for a release-batched bump.

Compatibility

• Existing shade entries continue to work unchanged. They were always at VERSION = 1, MINOR_VERSION = 0; this PR doesn't touch the schema or require migration.
• Existing users with a hand-edited const.HOME_KEY continue to work because the store-lookup falls back to const.HOME_KEY when no stored key exists. They can keep their setup or migrate to the store on first restart by running the extractor.
• Anyone with merged devices from bug Are you still using/developing this? #1 stays merged until they reset (the registry doesn't auto-unmerge). Their new entries register fresh MAC-based identifiers next to the bogus name-based ones, but the merge persists. I'm happy to add a one-shot cleanup utility if you want.

Testing

Verified against a 10-shade installation in Panama:

• 5 ACR rollers (Kitchen Big, Kitchen Small, Master Bedroom Roller, Guest Room Roller, Baby Room Roller) — model 49, sw 399
• 4 KDT curtain motors (Master Bedroom Curtains, Baby Room Curtains, Guest Room Curtains, KDT:4D59) — model unknown, sw 419 — these are the ones that strictly enforce encryption
• 1 KDT:B428 — also sw 419

Pre-fix: KDT curtains silently timed out on every open/close command from HA (TimeoutError: Device did not send confirmation from api._cmd) while ACR rollers worked. Hand-editing const.HOME_KEY made the KDTs work too, but the next HACS update wiped it.

Post-fix:

• Manual extraction flow tested via REST against gateway http://192.168.187.237. Returns the expected 16-byte key, persists to .storage/hunterdouglas_powerview_ble, aborts with home_key_saved.
• Zeroconf advertised type confirmed as _PowerView-G3._tcp.local. via dns-sd -B.
• Close → open cycle on one shade per room (guest bedroom, baby room, master bedroom, kitchen) all completed with HTTP 200 and state=open, pos=100 within ~30s, no TimeoutError in logs.
• Coordinator data_available gate verified: stale shades transition to unavailable after the 5-minute window.
• HA restart preserves the key and shades come back encrypted-capable without any manual step.

License

Submitted under the project's Apache 2.0 license per CONTRIBUTING.md.

---

Happy to split this into smaller PRs (e.g. bug fixes separately from the gateway flow) if that's easier to review. The fixes are independent — the only ordering constraint is that fix #4 needs to land with or after fix #2.

[Cloudore]

Pushed a small follow-up to the head branch: a01d95c removes the _target_position fallback from is_opening / is_closing in cover.py.

I left the fallback in the original PR because it gives slightly more responsive UI during the brief window between sending a BLE command and the first advert echoing the movement bits. Running it end-to-end with HomeKit driving sequential set_cover_position calls across six shades exposed the cost: _target_position never resets on successful command completion, and api.is_connected flickers as BLE connections come and go. Hours after the shade physically settles, the entity still reports is_opening / is_closing based on the stale target. The cached state field then freezes on the optimistic write (state="closed" with current_position=100 was the visible symptom in HomeKit) because the next state recomputation keeps returning a movement state instead of resolving to the actual position.

Both properties now use only the V2 advert movement bit. The shade is the source of truth for whether it is moving — the integration should not second-guess it. Reverified: every entity settled cleanly to a state matching its current_position, with no stale movement state lingering after the motors stopped.

Happy to revert if you want the responsive-UI tradeoff back; the brief blank window during a command is the only regression and adverts come every 1–2s so it is not noticeable in practice.

[Cloudore]

One more follow-up: 6a20683 exposes bit 0x04 of manufacturer-data byte 8 as a new diagnostic binary_sensor per shade — service_required (device_class=problem).

The bit was found while debugging two shades that wouldn't physically respond to set_cover_position. The shades:

• accepted BLE connections (connect took 0s)
• ACK'd the encrypted SET_POSITION frame with error_code=0 (the integration logged a clean round-trip)
• and yet never moved — subsequent adverts kept reporting position 0

Comparing the manufacturer data byte-by-byte against eight working shades, the only consistent diff was byte 8: 0xd6 on the broken pair vs 0xd2 on every working one — a single set bit at 0x04 that the integration didn't decode. The bit clears once the motor is re-calibrated via the PowerView app or driven through its full range with the physical remote.

Without this sensor the failure mode is silent: HA returns HTTP 200, no error is logged anywhere, and the cover entity just sits at the last reported position. Users mostly blamed the integration. Now the service_required flag flips on immediately and points at the real cause.

Setting entity_category=DIAGNOSTIC so it stays out of the dashboard for healthy installations. Open to a clearer name — needs_calibration, position_unknown — depending on how confident you are about the semantics; I named it service_required because the only thing I've definitively observed is "shade is in a state where it won't move."

[Cloudore]

Final follow-up: d9a2d4d makes the integration self-heal for shades in the low-power state.

After exposing the service_required bit (previous commit) and watching one curtain that had refused every HA command for hours, the pattern became clear: shades whose advert reports bit 0x04 of byte 8 set ACK SET_POSITION with error_code=0 but ignore it physically. Sending the same frame again ~500ms later in the same BLE session fixes it — the first frame wakes the motor without moving anything; the second frame, sent while the motor is briefly awake, actually drives it. This is what a physical remote effectively does (button press → motor wakes → command processed), and matches the empirical observation that the bit clears for ~10s during a remote-driven move.

The implementation is gated on service_required so it adds zero latency to healthy shades:

• api.set_position / open / close got a wake_first=False kwarg. When true, the SET_POSITION frame is sent twice with a 0.5s asyncio.sleep between, both inside the same BLE connection (disconnect=False on the first send). The wake-up swallows BleakError / TimeoutError so the real send still runs even if the wake itself failed.

• cover.py adds a _needs_wake property that returns self._coord.data.get("service_required") and passes it through to every command path. Shades without the bit set continue to get a single send.

Live verification on KDT:D079, which had been stuck for hours:

Pre-state:           state=closed pos=0  service_required=on
HK open_cover:       HTTP 200 in 0.7s   ← two-stage send including 0.5s gap
Post-state (+5s):    state=closing pos=47
Post-state (+30s):   state=open pos=100

And the debug log shows the second send correctly reusing the open connection:

21:45:25.011 KDT:D079 setting position to 100/... (wake_first)
21:45:25.011 Connecting KDT:D079
21:45:25.163 received BLE data: d2 27 8f 91 16
21:45:25.664 KDT:D079 already connected           ← 0.5s later
21:45:25.703 received BLE data: d2 27 8c 91 16
... advert byte 8 flips from 0xd6 to 0xd2; current_position climbs 0 → 37 → 42 → 100

Same test on shades with service_required=off (Kitchen Big and the 4 other working rollers) shows no wake-first overhead — single send, same latency as before this PR.

End-state for the original integration user pain: any shade that slips into low-power mode auto-recovers on the next command, no remote-press or PowerView app trip required.

[Cloudore]

Renamed: 85b7151 — service_required → low_power_mode across decode key, coordinator, cover._needs_wake, binary_sensor entity, translation. "Service required" implied the user needed to take action; with the wake_first send path landed in the previous commit, the integration auto-recovers and the bit is purely diagnostic. Also dropped device_class=problem — it isn't a problem the user has to respond to. entity_category=diagnostic stays.

Sorry for the rename mid-PR. If you'd rather squash all six commits before merging that's fine by me — happy to rebase.

[Cloudore]

One more: a3e97b4 — derive cover direction from position delta instead of trusting the advert's is_opening/is_closing bits.

Observed live in HomeKit: a KDT curtain (KDT:3F34) was being driven open by HA and HomeKit's accessory state displayed "Closing" the entire time the curtain was physically opening, then snapped to "Open" when the motor settled. Pulling the raw advert during that move:

KDT:D079 (during open command, position climbing 0 → 37)
manufacturer_data 2073 = b'"5E\xc9\x05\x00\x00\x00\xd2'
bytes[3:5] = 0xc9 0x05  → pos = 0x05c9 = 1481 → current_position = 37.0
pos & 0x3 = 1

pos & 0x3 == 1 is what the integration decodes as is_closing per dec_manufacturer_data. But the shade was opening — current_position went 0 → 37 → 95 → 100 over the next few seconds. So at least for some KDT firmwares, the encoding of the is_opening / is_closing bits is opposite from what the ACR rollers use. Trusting the bit produces the wrong state during the entire move; HomeKit dutifully renders it.

Position itself is unambiguous. After updating self.data with the decoded V2 fields, the coordinator now compares ATTR_CURRENT_POSITION against the previously-decoded value and overrides the two movement bits:

new > prev  → is_opening=True,  is_closing=False
new < prev  → is_opening=False, is_closing=True
new == prev → both False (motor stopped)

is_closed still drives the steady-state classification, so a settled shade reports open / closed correctly.

Verified post-fix on the same KDT:3F34: open command traces as closed/0 → opening/44 → opening/95 → open/100 instead of the previous closed/0 → closing/44 → closing/95 → open/100.

Trade-off: there is a one-advert latency before the first direction transition (the very first advert after the command has no _last_position to compare against, so movement bits stay False). In practice that is invisible because the next advert arrives within 1–2s with a non-zero position and the direction flips correctly.

If you know which shade families use which encoding I'm happy to make this a per-type lookup instead of a blanket override; otherwise this is the most robust fix that works across both flavors.

[Cloudore]

Major cleanup of the state machine: 8f20dcd. Replaces the position-delta direction inference from the previous commit with a target-vs-current model that's noise-free and matches user intent, plus kills the ghost state transitions.

Why the rewrite: the previous position-delta approach (commit a3e97b4) still flickered in practice because with multiple BLE proxies in one home, the same shade's adverts arrive interleaved out of order — even position bounces as small as ±3% produce visible state flips during a smooth open. Threshold filtering didn't help; the threshold needed to filter noise was the same magnitude as a normal advert delta. The cleaner approach is to derive direction from command intent.

Direction (cover.py)

is_opening / is_closing now read _target_position (set by every async_set_cover_position / async_open_cover / async_close_cover) and compare against current_cover_position:

DIRECTION_DEADZONE = 2
TARGET_TTL = 60.0  # seconds since last command

@property
def is_opening(self):
    if not self._target_is_fresh():
        return False
    if self.current_cover_position is None:
        return False
    return self._target_position > self.current_cover_position + self.DIRECTION_DEADZONE

Symmetric for is_closing. _target_is_fresh() returns False after TARGET_TTL seconds so a stale command can't make the entity report "opening" forever once the user has moved the shade with the physical remote.

Endpoint deadzone (cover.py)

is_closed uses the same deadzone:

@property
def is_closed(self) -> bool | None:
    pos = self.current_cover_position
    if pos is None:
        return None
    return pos <= CLOSED_POSITION + self.DIRECTION_DEADZONE

Stops the "open"/"closed" bounce when a motor settles at position 1 or 2 instead of exactly 0 (common end-of-travel drift). Also correctly returns None for unknown rather than falsely reporting "open".

Coordinator (coordinator.py)

The position-delta override is removed entirely. The coordinator still clears the advert's is_opening / is_closing bits per event (they're unreliable — KDT curtains and ACR rollers encode them inversely) so the cover entity falls through to the target-based properties.

Optimistic state write removed (cover.py)

The three command handlers no longer call async_write_ha_state() after the BLE round-trip. The PassiveBluetoothCoordinatorEntity update path already writes state on every BLE advert. The post-BLE optimistic write was racy: a stale-position advert landing during the ~500ms BLE window made the entity write inconsistent state (closed → closing → closed ghosts at the start of a close on a near-closed shade). Removing it eliminates the ghosts; the first real BLE advert after the command drives the visible transition.

Drain queue (api.py)

While I was in here: _cmd now loops to drain _cmd_next inside the lock. Previously, a second command that landed while a BLE write was in flight (e.g. user taps close in HomeKit while the shade is still opening) was silently dropped — the in-flight write completed, the lock released, and the queued command never ran. Now the lock-holder picks up the newer command on the same BLE session and sends it, so the second tap reliably interrupts the first. ~0.5s perceived latency from the queue drain plus the second BLE round-trip.

Live verification

Baby Room Roller (cover.acr_a67c_shade), full open cycle from closed/0:

+0.5s   state=open       pos=0   (one transient frame at the very start)
+1.0s   state=opening    pos=0
+1.5s   state=opening    pos=3
+5.0s   state=opening    pos=20
+10.0s  state=opening    pos=44
+15.0s  state=opening    pos=69
+20.0s  state=opening    pos=94
+21.0s  state=open       pos=100

Smooth position climb every 0.5s, single direction state throughout the move, settles to open at the target. Same shape for closes.

The one-frame open at +0.5s is an artifact of the entity getting an early BLE advert before async_set_cover_position's _set_target runs — could be eliminated by reordering _set_target before the early-return check, but it's a single transient and HomeKit doesn't display it.

[Cloudore]

Found the root cause of the "Closing..." while opening report in HomeKit: 81c2af8. The previous commit removed the post-BLE optimistic write to kill ghost state transitions, which exposed an interaction with HA's HomeKit Bridge I hadn't accounted for.

In homeassistant.components.homekit.type_covers.OpeningDevice.async_update_state:

if isinstance(current_position, (float, int)):
    current_position = int(current_position)
    self.char_current_position.set_value(current_position)
    # Writing target_position on a moving cover
    # will break the moving state in HK.
    if new_state.state not in MOVING_STATES:
        self.char_target_position.set_value(current_position)

Every time HA reports the cover in a non-moving state, the Bridge clobbers HK's TargetPosition characteristic to match CurrentPosition. iOS HomeKit derives the direction label from (Target - Current), not from PositionState, so once Target gets reset to 0 (current position when shade was closed and command first lands), every subsequent CurrentPosition update of 1, 2, ..., 100 reads as "going down from current toward target=0", which iOS displays as "Closing..." — for the entire physical open.

The window for the Bridge to clobber Target is small but real: between the cover entity's async_set_cover_position running and the first BLE advert that triggers a state recomputation in a moving state, the entity passes through "open" or "closed" briefly (target set, but no advert-driven state write yet). HA's state machine fires a state-changed event for that frame, the Bridge processes it, and Target is lost.

Fix is to write state before the BLE await:

self._set_target(round(target_position))
self.async_write_ha_state()   # <-- new: state="opening"/"closing" immediately
try:
    await self._coord.api.set_position(round(target_position), wake_first=self._needs_wake)
except BleakError as err:
    ...

With _target_position set, is_opening / is_closing already report the right direction (they use target-vs-current), so the entity immediately enters the correct moving state. The Bridge never sees a non-moving frame during the move, never resets Target, iOS displays the user's intent.

The post-BLE write is still removed — only the pre-BLE write was needed.

Live verification

Master Bedroom Curtains, full open from closed:

+0.5s   state=opening pos=0      <-- written immediately, before BLE round-trip
+1.0s   state=opening pos=2
+5.0s   state=opening pos=45
+10.0s  state=opening pos=93
+11.0s  state=open    pos=98     <-- within DIRECTION_DEADZONE of target=100
+11.5s  state=open    pos=100

No transient "open" frame between command and "opening". HomeKit no longer sees the cover enter a non-moving state mid-command, so the Bridge keeps the user's TargetPosition intact and iOS displays "Opening..." throughout the move (and "Closing..." for closes).

[patman15]

Isn't that PR similar to #29 ?

[Cloudore]

Isn't that PR similar to #29 ?

Sorry if it is, I just went through making the experience "Native" with Bluetooth Proxies and HomeKit and did not look at the PRs, just iterated (with Claude, admittedly) through versions until the whole thing felt less shaky. My setup feels solid now.

[safepay]

Isn't that PR similar to #29 ?

I don't have any problems with people repeating my work in AI. I might just separate my work in to a separate repository as it is a refactor that does not allow migration. Given the need to handle the key, I simply choose a more standard HA method for doing this that is the same as other integrations with similar requirements.

I also assumed that there are not that many blinds in a house (vs smart switches), so recreating them would not be a problem.

Just let me know if you want to take the refactor approach that sets up a "virtual hub" for management and auto-discovery or go with this iterative approach that retains the current architcture.

[patman15]

Ok, I need to take a look, to see the delta of both PRs. In general I do not trust AI on good architecture. 😎

[safepay]

Ok, I need to take a look, to see the delta of both PRs. In general I do not trust AI on good architecture. 😎

The difference lies fundamentally in the architecture. I took the approach of getting the architetecture aligned with more modern HA practices and sacrificing the migration option given the early state of the repository to try and allow myself and others to collaborate on a proven architectural base, while this PR fixes the outstanding issues but sacrifices long-term structural maintainability.