Skip to content

feat(pool): transparent reconnect on stale connection - #55

Merged
Irishsmurf merged 1 commit into
mainfrom
feat/pool-transparent-reconnect
Jun 13, 2026
Merged

feat(pool): transparent reconnect on stale connection#55
Irishsmurf merged 1 commit into
mainfrom
feat/pool-transparent-reconnect

Conversation

@Irishsmurf

@Irishsmurf Irishsmurf commented Jun 13, 2026

Copy link
Copy Markdown
Owner

Addresses #36.

Problem

When a persistent TCP connection goes stale (e.g. because of a device reboot or network blip), the connection pool raised DLightConnectionError and required the caller to handle reconnecting and retrying manually.

Solution

  • Introduced ReconnectingState, ReconnectingStreamReader, and ReconnectingStreamWriter proxy wrappers inside the private ConnectionPool (_pool.py).
  • When a connection retrieved from the pool encounters a connection-level error (OSError or asyncio.IncompleteReadError) during read/write/drain operations:
    1. The stale connection is closed and discarded.
    2. A new connection is transparently established.
    3. The failed command is re-sent (replaying buffered bytes) and retried once.
  • Failures on brand-new (not reused) connections propagate directly to avoid masking real errors.
  • Documented this transparent reconnect behavior in ConnectionPool's docstring.
  • Updated documentation files: docs/architecture.md, docs/ARCHITECTURE.md, and docs/user-guide/connections.md.

Tests

  • Added test_transparent_reconnect_on_stale_connection to cover connection drops mid-session using FakeDLightServer.
  • Added test_no_transparent_reconnect_on_fresh_connection_failure to verify that new connections do not retry on failure.
  • Confirmed all 124 unit and regression tests pass.

@Irishsmurf Irishsmurf added the enhancement New feature or request label Jun 13, 2026
@Irishsmurf Irishsmurf added this to the v2.0.0 milestone Jun 13, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements transparent automatic reconnection in ConnectionPool for stale persistent connections, retrying a failed command once on a new connection before raising an error. It introduces proxy classes ReconnectingStreamReader and ReconnectingStreamWriter to wrap the underlying stream reader and writer, along with corresponding documentation and regression tests. The review feedback suggests several improvements: unpacking is_reused directly from _checkout instead of using getattr, wrapping asyncio.TimeoutError in DLightTimeoutError during drain() for consistency, and simplifying the exception handling in ReconnectingStreamReader.readuntil by removing asyncio.LimitOverrunError from the caught exceptions tuple.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread dlightclient/_pool.py Outdated
Comment on lines +225 to +227
reader, writer = await self._checkout(key, host, port, ssl, connect_timeout)
is_reused = getattr(writer, "_is_reused", False)
state = ReconnectingState(self, host, port, ssl, connect_timeout, is_reused, reader, writer)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the call to _checkout to unpack the new 3-tuple return value containing is_reused, avoiding the need to retrieve it via getattr.

Suggested change
reader, writer = await self._checkout(key, host, port, ssl, connect_timeout)
is_reused = getattr(writer, "_is_reused", False)
state = ReconnectingState(self, host, port, ssl, connect_timeout, is_reused, reader, writer)
reader, writer, is_reused = await self._checkout(key, host, port, ssl, connect_timeout)
state = ReconnectingState(self, host, port, ssl, connect_timeout, is_reused, reader, writer)

Comment thread dlightclient/_pool.py
Comment on lines +79 to +87
if self.write_buffer:
try:
self.writer.write(self.write_buffer)
await asyncio.wait_for(self.writer.drain(), timeout=self.connect_timeout)
except Exception as e:
_LOGGER.error(f"Transparent reconnect failed during write/drain: {e}")
raise DLightConnectionError(
f"Network error during transparent reconnect write/drain: {e}"
) from e

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If the drain() operation times out during a transparent reconnect, it raises asyncio.TimeoutError. This should be wrapped in DLightTimeoutError for consistency with the rest of the library's timeout handling (e.g., in client.py), rather than being caught by the generic except Exception block and wrapped in DLightConnectionError.

Suggested change
if self.write_buffer:
try:
self.writer.write(self.write_buffer)
await asyncio.wait_for(self.writer.drain(), timeout=self.connect_timeout)
except Exception as e:
_LOGGER.error(f"Transparent reconnect failed during write/drain: {e}")
raise DLightConnectionError(
f"Network error during transparent reconnect write/drain: {e}"
) from e
if self.write_buffer:
try:
self.writer.write(self.write_buffer)
await asyncio.wait_for(self.writer.drain(), timeout=self.connect_timeout)
except asyncio.TimeoutError as e:
_LOGGER.error(f"Transparent reconnect timed out during write/drain: {e}")
raise DLightTimeoutError(
f"Timeout during transparent reconnect write/drain"
) from e
except Exception as e:
_LOGGER.error(f"Transparent reconnect failed during write/drain: {e}")
raise DLightConnectionError(
f"Network error during transparent reconnect write/drain: {e}"
) from e

Comment thread dlightclient/_pool.py Outdated
Comment on lines +174 to +175
except (OSError, asyncio.IncompleteReadError, asyncio.LimitOverrunError) as e:
if not isinstance(e, asyncio.CancelledError) and not isinstance(e, asyncio.LimitOverrunError):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In ReconnectingStreamReader.readuntil, asyncio.LimitOverrunError is caught in the except block, only to be explicitly checked and propagated without retrying. Since LimitOverrunError is not a subclass of OSError or IncompleteReadError, omitting it from the except tuple will allow it to propagate naturally, simplifying the code.

Suggested change
except (OSError, asyncio.IncompleteReadError, asyncio.LimitOverrunError) as e:
if not isinstance(e, asyncio.CancelledError) and not isinstance(e, asyncio.LimitOverrunError):
except (OSError, asyncio.IncompleteReadError) as e:
if not isinstance(e, asyncio.CancelledError):

@Irishsmurf
Irishsmurf force-pushed the feat/pool-transparent-reconnect branch from 57e0529 to d12e420 Compare June 13, 2026 22:23
@Irishsmurf
Irishsmurf force-pushed the feat/pool-transparent-reconnect branch from d12e420 to 1614272 Compare June 13, 2026 22:24
@Irishsmurf
Irishsmurf merged commit 3ddf467 into main Jun 13, 2026
5 checks passed
@Irishsmurf
Irishsmurf deleted the feat/pool-transparent-reconnect branch June 13, 2026 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant