Skip to content

Named sessions and etserver restart survival - #793

Merged
MisterTea merged 6 commits into
MisterTea:masterfrom
epsalmond:named-sessions
Sep 26, 2026
Merged

MisterTea merged 6 commits into
MisterTea:masterfrom
epsalmond:named-sessions

Conversation

@epsalmond

@epsalmond epsalmond commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Sessions get a name, persist across client crashes and reboots, and survive etserver restarts.

I started this work because ubuntu was killing all my sessions at 3am doing updates. Then continued because (surprise!) there was a memory leak in node eating all the memory on my laptop. And an NFS bug in osx was rebooting me.

For the last month, having ~20 sessions open across tabs in two ghostty windows has been my primary workflow.

QoL improvements have happened along the way including being noisier when stealing a session from an active session. Further improvements could be detecting and cleaning up stale sessions.

Client

  • et --name <n> creates or reattaches a named session. Direct connections persist by default with a short generated name; --no-persist opts out.
  • et --list shows saved sessions with their captured terminal title and last heartbeat. --attach reattaches, --kill terminates a named session on the server.
  • Session files live in ~/.et/sessions/ with owner-only permissions, atomic no-clobber writes, and SSH setup errors no longer echo server's output to prevent passkey leakage

Protocol

  • Reset handshake for fresh client reattachment. Authenticated by challenge/response and backwards compatible. Fresh keys per reset.
  • Additive proto fields only. Protocol version is unchanged and the backwards-compatibility test passes. 🤘

Server

  • etterminal keeps the pty alive when the router drops and re-registers with bounded backoff, so systemctl restart et no longer kills sessions.
  • A restart grace window answers reconnects with retry-later instead of a hard rejection. Reverse tunnels dropped by a restart are reported.
  • Shells survive a stopped etserver indefinitely. Say it plainly: "etterminal retries the router until it returns, so systemctl stop et leaves shells running until they exit or --disconnect-timeout closes them."
  • -T pipe sessions are never saved and end on restart.
  • The two master bugs fixed while rebasing: a re-registered terminal never started the feat: close disconnected etterminal sessions after a timeout #855 disconnect deadline, and the feat: positional remote command #850 parser lost the host after the jumphost flag.

Tests

I'm working on resolving these:

High:

  • Reset recovery can be triggered by a known client ID without passkey proof, allowing disruption/data loss. done

Medium:

  • Windows router restart and --kill behavior are incomplete. done
  • Restart during initial Unix setup is not recovered. done
  • external/UniversalStacktrace moves backward to an ancestor commit. done
  • name can silently ignore newly requested forwarding options. done
  • Fixed /tmp paths and broad pkill patterns make some shell tests unsafe for concurrent runs. done

Rejected:

  • Reset recovery is incompatible with older peers while the protocol version remains unchanged. I'm rejecting this one because this only matters when etserver restarts and comes up as the new version. Before this PR, etserver loses its keys and clients are dropped anyway. So in practice this can't be reached.

@codecov

codecov Bot commented Aug 20, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.15152% with 589 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.71%. Comparing base (f3137ce) to head (60b0b68).

Files with missing lines Patch % Lines
src/terminal/SessionStore.cpp 62.70% 75 Missing and 41 partials ⚠️
src/terminal/TerminalClient.cpp 44.08% 36 Missing and 16 partials ⚠️
test/integration_tests/PseudoUserTerminalTest.cpp 60.15% 26 Missing and 25 partials ⚠️
src/terminal/UserTerminalHandlerUnix.cpp 57.26% 32 Missing and 18 partials ⚠️
src/terminal/PseudoUserTerminalUnix.hpp 36.92% 25 Missing and 16 partials ⚠️
src/terminal/TitleParser.cpp 67.21% 24 Missing and 16 partials ⚠️
src/terminal/TerminalServer.cpp 76.11% 12 Missing and 20 partials ⚠️
test/integration_tests/RouterRestartTest.cpp 93.45% 13 Missing and 15 partials ⚠️
test/unit_tests/SessionStoreTest.cpp 90.31% 13 Missing and 15 partials ⚠️
src/base/CryptoHandler.cpp 79.04% 11 Missing and 11 partials ⚠️
... and 14 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #793      +/-   ##
==========================================
+ Coverage   78.86%   79.71%   +0.84%     
==========================================
  Files         143      150       +7     
  Lines       18176    21233    +3057     
  Branches    11619    13775    +2156     
==========================================
+ Hits        14334    16925    +2591     
- Misses       2303     2559     +256     
- Partials     1539     1749     +210     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@epsalmond epsalmond closed this Aug 20, 2026
@MisterTea

Copy link
Copy Markdown
Owner

@epsalmond Why close this?

@epsalmond

Copy link
Copy Markdown
Contributor Author

@MisterTea is was vibe-opened and wasn't ready.

I've since made a ton of improvements, it actually works as intended, and it's my full-time terminal workflow. If you're interested in it I'll re-open it with a month worth's of updates and bugfixes.

@MisterTea

Copy link
Copy Markdown
Owner

Sure, go for it

@MisterTea MisterTea reopened this Sep 17, 2026
@epsalmond epsalmond changed the title Named sessions: --name/--list/--attach, session persistence, reset handshake Named sessions and etserver restart survival Sep 18, 2026
@epsalmond
epsalmond force-pushed the named-sessions branch 4 times, most recently from e50d130 to 7a4c4b7 Compare September 19, 2026 02:56
@Kronuz

Kronuz commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Nice work on this PR. Restart survival and the reset handshake are real gaps for sessions. Just flagging one thing: #819 also adds a client-side --attach NAME under ~/.et/sessions/ (written before I'd seen this PR), so the flag name and directory overlap with what you've built here.

The one in #819 is a much thinner piece (just a cached reconnect id/passkey with no restart survival and no title capture), and is needed for etctl control plane (#820). When this PR lands I'd like to rework the named sessions there onto your more complete SessionStore.

@epsalmond

Copy link
Copy Markdown
Contributor Author

@Kronuz Thanks! It has been battle-tested.

I had to stand up a windows vm to work through some of the stuff that has landed since I first opened this, and today there are a few merge conflicts. I expect I'll be ready to merge today.

epsalmond added a commit to epsalmond/EternalTerminal that referenced this pull request Sep 23, 2026
@MisterTea
MisterTea requested a balanced review from Copilot September 23, 2026 14:14

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@epsalmond
epsalmond force-pushed the named-sessions branch 2 times, most recently from 84d638c to 241fad5 Compare September 24, 2026 03:02
@epsalmond

Copy link
Copy Markdown
Contributor Author

@MisterTea @Kronuz I think this is ready.

Freebsd test failed this run but it looks flakey. PR is updated. Protocol backwards-compatible (so no bump) and I cleaned up the commits and comments. Rebased on top of #850 and #855.

Two bugfixes for master included: a terminal that re-registers after a restart never started the #855 disconnect deadline, and the #850 positional-command parser lost the host after the jumphost flag.

I don't have anything for the control plane in this PR but I'm happy to do a follow up on top of #819 to add etserver --list.

@MisterTea

Copy link
Copy Markdown
Owner

@Kronuz if this PR lands, what's the state of your PRs? Would you rebase on this?

@psennats

Copy link
Copy Markdown

I want this too, let me know how I can be of any help.

@Kronuz

Kronuz commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

@MisterTea Yes, and I've already done the rebase: both of my PRs are rebased onto 793's head and pushed, as et-session-support-793 and etctl-binary-793 on my Kronuz/EternalTerminal fork. Rebasing makes my #819 smaller and cleaner. If we land this one I can update my two PRs.

A client resuming a session from a fresh process has no sequence
history, so both sides can now reset their state and rekey from a
server-chosen salt. The server honors a reset only after a keyed
challenge, offered to clients that set supportsChallenge, and proves its
decision back. Plain connects never ask for a reset. PROTOCOL_VERSION
stays 6; older peers get the original handshake, and that legacy path
goes away at the next version bump.
@epsalmond

Copy link
Copy Markdown
Contributor Author

Main moved out from under me again. Should I rebase again? (jk I already am).

Excited to get this in but let me know if I should hold off, or let me know when this is up next and I'll wait.

et saves each direct session's endpoint, id, and passkey under
~/.et/sessions so --attach can resume it from a new process with a
reset; --list and --kill manage the records. Jumphost and -T sessions
are not saved. SshSetupHandler no longer echoes server output in its
errors, since that output can contain the credentials; this drops the
old hint about a .bashrc that prints.
When the router goes away, etterminal keeps its shell and re-registers
with the same credentials, retrying indefinitely with backoff capped at
30s, and the new server resumes the session with a reset. For 60s after
startup the server answers unknown ids with RETRY_LATER so clients wait
for re-registration. -T sessions end instead, since their packet-framed
stream cannot be resumed. Stopping etserver leaves shells running until
they exit or --disconnect-timeout closes them.
@epsalmond

Copy link
Copy Markdown
Contributor Author

Rebased again, should be clean to merge.

jasongauci-webai and others added 3 commits September 25, 2026 22:20
Keep reset recovery together with catchup ordering, OpenSSH short flags,
and the per-session disconnect timeout so the branch can merge.

Co-authored-by: Cursor <cursoragent@cursor.com>
etterminal re-advertises the client's timeout when it re-registers, so a resumed pump does not fall back to the server global.

Co-authored-by: Cursor <cursoragent@cursor.com>
Raw terminal bytes were parsed as a packet length, which dropped the session and failed the Windows vcpkg tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
@MisterTea
MisterTea merged commit 540367a into MisterTea:master Sep 26, 2026
44 of 45 checks passed
@Kronuz

Kronuz commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Nice! 🎉

I’ve rebased and updated my two pull requests (#819 and #820) on top of this.

@psennats

Copy link
Copy Markdown

Any way to test this without having to build it myself? Maybe a dev nightly build somewhere? Sorry if the questions is already answered elsewhere...

@MisterTea

MisterTea commented Sep 27, 2026 via email

Copy link
Copy Markdown
Owner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants