Fleet agent: don't let a dropped connection kill the launcher - #169
Open
rudysev wants to merge 1 commit into
Open
Fleet agent: don't let a dropped connection kill the launcher#169rudysev wants to merge 1 commit into
rudysev wants to merge 1 commit into
Conversation
A phone browser opening the remote page crashed the launcher on a Portal 10"
gen2 (Android 10). Browsers preconnect speculatively and abandon connections
when a tab is backgrounded, so FleetHttpServer.readHead() would block until the
30s soTimeout and throw SocketTimeoutException. serve() ran directly as a pool
task, and a checked exception escaping a Runnable is wrapped by
ThreadPoolExecutor.runWorker into java.lang.Error — reaching the thread's
uncaught handler:
FATAL EXCEPTION: pool-2-thread-2
java.lang.Error: java.net.SocketTimeoutException: Read timed out
at com.immortal.launcher.FleetHttpServer.readHead(FleetHttpServer.kt:239)
CrashGuard then recovered the process, so enabling "remote control from phone"
and loading the page bounced the launcher. Every other failure path in the file
was already inside runCatching; the header read was the one hole.
- serveSafely() wraps serve(): timeouts and resets log at debug, anything else
warns, and the socket is always closed.
- Split the read timeout — 10s waiting for a request line, 30s once a request is
in flight (uploads keep the long budget). An abandoned connection previously
pinned a pool thread for the full 30s.
- Pool 4 -> 8. With `Connection: close` every asset and poll costs a fresh
connection and browsers fan out ~6 in parallel, so the page queued behind
itself.
FleetHttpServerSocketTest drives a real loopback server: reset connections stay
off the uncaught handler and the pool keeps serving, plus preflight, malformed
request, throwing handler, and stop(). Unit tests need
`unitTests.isReturnDefaultValues` because the off-device android.util.Log stub
throws "not mocked", which put every logging path out of reach of a JVM test.
Verified on a Portal 10" gen2 (Android 10): held idle connections past the
header timeout, same exception now logs as
`D ImmortalFleet: idle connection timed out: Read timed out`, PID unchanged, and
the next request answered normally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Loading the phone remote page crashed the launcher.
FleetHttpServerran each connection directly as a pool task:pool.execute { serve(socket) }Browsers preconnect speculatively and abandon connections when a tab is backgrounded, so
readHead()blocked until the 30ssoTimeoutand threwSocketTimeoutException. A checked exception escaping aRunnablegets wrapped byThreadPoolExecutor.runWorkerintojava.lang.Error, which reaches the thread's uncaught handler:CrashGuardthen recovered the process, so turning on "remote — control from phone" and opening the page from a phone browser bounced the launcher. Every other failure path in the file was already insiderunCatching; the header read was the one hole.Changes
serveSafely()wrapsserve()— timeouts and resets log at debug, anything else warns, and the socket is always closed. A dead connection can no longer take the process down.Connection: close, every asset and poll costs a fresh connection and browsers fan out ~6 in parallel, so the remote page could queue behind itself.Testing
FleetHttpServerSocketTestdrives a real loopback server: reset connections stay off the uncaught handler and the pool keeps serving, plus coverage for preflight, a malformed request, a throwing handler, andstop(). Full suite passes (298 tests).The unit-test change to
app/build.gradle.kts(unitTests.isReturnDefaultValues) is needed because the off-deviceandroid.util.Logstub throws "not mocked", which put every logging path out of reach of a JVM test. No test regressions.Verified on a Portal 10" gen2 (Android 10) with the
devbuild: held idle connections open past the header timeout, and the same exception now surfaces aswith the PID unchanged and the next request answered normally. The phone remote is usable end to end.
🤖 Generated with Claude Code