Skip to content

proxy: add local connection limit to ListenConnections#269

Merged
ryanofsky merged 3 commits into
bitcoin-core:masterfrom
enirox001:04-26-ipc-local-connection-limit
Jul 8, 2026
Merged

proxy: add local connection limit to ListenConnections#269
ryanofsky merged 3 commits into
bitcoin-core:masterfrom
enirox001:04-26-ipc-local-connection-limit

Conversation

@enirox001

@enirox001 enirox001 commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

This adds an optional local connection limit toListenConnections().

Previously, ListenConnections() would accept incoming connections indefinitely. This branch adds an optional max_connections parameter so a listener can stop accepting new connections once a per-listener cap is reached, and resume accepting when an existing connection disconnects.

The limit is local to the listener instead of global to the EventLoop. This keeps the state and behavior scoped to the listening socket, and is closer to the direction discussed downstream for per--ipcbind limits.

This also adds a test covering the behavior with max_connections=1, verifying that:

  • the first client is accepted normally
  • a second client is not accepted while the first remains connected
  • the second client is accepted after the first disconnects

Note This PR includes a major version bump to v12 due to the API addition. If #274 lands earlier and bumps the version to v12 first, we will need to bump the version here again.

@DrahtBot

DrahtBot commented Apr 8, 2026

Copy link
Copy Markdown

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline and AI policy for information on the review process.

Type Reviewers
ACK ryanofsky
Stale ACK xyzconstant, Eunovo

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

Conflicts

No conflicts as of last run.

@enirox001 enirox001 marked this pull request as draft April 8, 2026 10:01
@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch 3 times, most recently from 3ef8e5c to 84ed607 Compare April 8, 2026 10:53

@ryanofsky ryanofsky left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach ACK 84ed607. Implementation of local connection limit here looks almost exactly like I would have expected.

I do think it would be helpful to see a draft PR in the bitcoin repo using this API (it should be fine to make libmultiprocess changes there and let the lint CI job fail so you don't need to mess with subtrees) because the approach in bitcoin/bitcoin#34978 of adding a global connection limit option isn't exactly compatible with the implementation here of implementing a per-address connection limit.

I'd personally prefer using per-address limits over introducing a global limit but both approaches seem reasonable

Comment thread include/mp/proxy-io.h Outdated
Comment thread include/mp/proxy-io.h Outdated
kj::Own<kj::ConnectionReceiver> listener;
std::optional<size_t> max_connections;
size_t active_connections{0};
bool accept_pending{false};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "proxy: add local connection limit to ListenConnections" (84ed607)

Curious if this accept_pending variable is actually necessary or if code could just compare active_connections and max_connections when deciding whether to listen. Would prefer to avoid redundancy in the state representation if possible even if makes individual checks more a little more verbose.

If accept_pending really is necessary would be a good to have a short comment about why.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept accept_pending, but added a short comment explaining why it is needed here. active_connections only counts accepted connections, so without a separate flag nested _Listen() calls could post multiple pending accept() calls before active_connections is incremented.

@xyzconstant xyzconstant May 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I tested the code without accept_pending field and hence its check at _Listen removed, the tests passed with no issues. Then, asked Claude to generate a test that exercises it and produced this:

diff --git a/test/mp/test/listen_tests.cpp b/test/mp/test/listen_tests.cpp
index b367938..78298c7 100644
--- a/test/mp/test/listen_tests.cpp
+++ b/test/mp/test/listen_tests.cpp
@@ -24,8 +24,10 @@
 #include <string>
 #include <sys/socket.h>
 #include <sys/un.h>
+#include <kj/exception.h>
 #include <thread>
 #include <unistd.h>
+#include <vector>
 
 namespace mp {
 namespace test {
@@ -112,11 +114,39 @@ public:
     std::thread thread;
 };
 
+//! kj::ExceptionCallback that captures KJ_LOG output into an external sink.
+//! Must be instantiated on the thread whose KJ logs you want to capture; it
+//! installs itself onto that thread's ExceptionCallback stack via its base
+//! constructor and removes itself in the destructor.
+class CaptureLogCallback : public kj::ExceptionCallback
+{
+public:
+    CaptureLogCallback(std::mutex& mu, std::string& sink) : m_mu(mu), m_sink(sink) {}
+
+    void logMessage(kj::LogSeverity severity, const char* file, int line, int contextDepth,
+                    kj::String&& text) override
+    {
+        {
+            std::lock_guard<std::mutex> lock(m_mu);
+            m_sink.append(text.cStr(), text.size());
+            m_sink.push_back('\n');
+        }
+        // Still let the default callback emit to stderr so test debug output
+        // isn't silenced for other observers.
+        kj::ExceptionCallback::logMessage(severity, file, line, contextDepth, kj::mv(text));
+    }
+
+private:
+    std::mutex& m_mu;
+    std::string& m_sink;
+};
+
 class ListenSetup
 {
 public:
     explicit ListenSetup(std::optional<size_t> max_connections = std::nullopt)
         : capped_listener(max_connections.has_value()), thread([this, max_connections] {
+              CaptureLogCallback log_capture(captured_log_mutex, captured_log);
               EventLoop loop("mptest-server", [this](mp::LogMessage log) {
                   if (log.level == mp::Log::Raise) throw std::runtime_error(log.message);
                   if (log.message.find("IPC server: socket connected.") != std::string::npos) {
@@ -144,6 +174,18 @@ public:
 
     ~ListenSetup()
     {
+        forceShutdown();
+        thread.join();
+    }
+
+    //! Synchronously tear down the event loop's task set so any pending accept
+    //! promises are destroyed now (rather than when the destructor runs later).
+    //! This makes it possible to assert on captured KJ log output before the
+    //! ListenSetup goes out of scope. Idempotent.
+    void forceShutdown()
+    {
+        if (shutdown_done) return;
+        shutdown_done = true;
         if (capped_listener) {
             EventLoop* loop;
             {
@@ -152,7 +194,6 @@ public:
             }
             if (loop) loop->sync([&] { loop->m_task_set.reset(); });
         }
-        thread.join();
     }
 
     size_t ConnectedCount()
@@ -184,11 +225,15 @@ public:
     UnixListener listener;
     std::promise<void> ready_promise;
     bool capped_listener{false};
+    bool shutdown_done{false};
     std::mutex counter_mutex;
     std::condition_variable counter_cv;
     EventLoop* event_loop{nullptr};
     size_t connected_count{0};
     size_t disconnected_count{0};
+    //! KJ log output captured from the server thread via CaptureLogCallback.
+    std::mutex captured_log_mutex;
+    std::string captured_log;
     std::thread thread;
 };
 
@@ -245,6 +290,34 @@ KJ_TEST("ListenConnections keeps capped listeners alive before reaching the limi
     KJ_EXPECT(client2->client->add(2, 3) == 5);
 }
 
+// Without `accept_pending`, cascaded close handlers each post a duplicate
+// accept(). KJ silently serializes them so the cap isn't exceeded, but the
+// extra pending promises are destroyed at cleanup and logged as
+// "PromiseFulfiller was destroyed without fulfilling the promise."
+// This test fails when accept_pending is removed and passes when it's intact.
+KJ_TEST("ListenConnections does not leak accept promises during disconnect burst")
+{
+    constexpr size_t kCap = 2;
+    ListenSetup setup(/*max_connections=*/kCap);
+
+    std::vector<std::unique_ptr<ClientSetup>> filling;
+    filling.reserve(kCap);
+    for (size_t i = 0; i < kCap; ++i) {
+        filling.push_back(std::make_unique<ClientSetup>(setup.listener.Connect()));
+    }
+    setup.WaitForConnectedCount(kCap);
+
+    filling.clear();
+    setup.WaitForDisconnectedCount(kCap);
+
+    // Trigger m_task_set.reset() now so any leaked accept promises get destroyed
+    // before we read captured_log.
+    setup.forceShutdown();
+
+    std::lock_guard<std::mutex> lock(setup.captured_log_mutex);
+    KJ_EXPECT(setup.captured_log.find("PromiseFulfiller was destroyed") == std::string::npos);
+}
+
 } // namespace
 } // namespace test
 } // namespace mp

Basically what this does is install a kj::ExceptionCallback in the server thread to capture the log "PromiseFulfiller was destroyed" generated by KJ runtime if cascaded disconnects each call _Listen and post a fresh accept() promise. The test fails with accept_pending removed and pass with it back.

@xyzconstant xyzconstant May 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it's not so obvious why this field is needed here, this patch basically guarantee the same outcome and also pass the test generated by Claude above:

diff --git a/include/mp/proxy-io.h b/include/mp/proxy-io.h
index 78924c6..c002811 100644
--- a/include/mp/proxy-io.h
+++ b/include/mp/proxy-io.h
@@ -851,11 +851,6 @@ struct ListenState
     kj::Own<kj::ConnectionReceiver> listener;
     std::optional<size_t> max_connections;
     size_t active_connections{0};
-    //! Tracks whether accept() has already been posted. This is needed because
-    //! active_connections only counts accepted connections, so without a
-    //! separate flag, nested _Listen() calls could queue multiple pending
-    //! accepts before active_connections increases.
-    bool accept_pending{false};
 };
 
 template <typename InitInterface, typename InitImpl>
@@ -866,9 +861,12 @@ void _ServeAccepted(EventLoop& loop, InitImpl& init, const std::shared_ptr<Liste
 {
     ++state->active_connections;
     _Serve<InitInterface>(loop, kj::mv(stream), init, [&loop, &init, state] {
+        const bool was_at_cap = state->max_connections && state->active_connections == *state->max_connections;
         assert(state->active_connections > 0);
         --state->active_connections;
-        _Listen<InitInterface>(loop, init, state);
+        if (was_at_cap) {
+            _Listen<InitInterface>(loop, init, state);
+        }
     });
 }
 
@@ -885,15 +883,12 @@ inline std::unique_ptr<EventLoopRef> _MakeCappedListenerRef(EventLoop& loop, con
 template <typename InitInterface, typename InitImpl>
 void _Listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<ListenState>& state)
 {
-    if (state->accept_pending) return;
     if (_ListenAtCapacity(*state)) return;
 
-    state->accept_pending = true;
     auto* ptr = state->listener.get();
     auto accept_ref{_MakeCappedListenerRef(loop, *state)};
     loop.m_task_set->add(ptr->accept().then(
         [&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable {
-            state->accept_pending = false;
             _ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream));
             _Listen<InitInterface>(loop, init, state);
         }));

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a better invariant than tracking accept_pending.

There should already be one pending accept whenever the capped listener is below capacity, and disconnects only need to post a new accept when they transition the listener from full to below full, because that is the only state where nocaccept was pending.

So checking this before decrementing active_connections avoids the duplicate-accept case without adding another state variable.

Taken and decided to use resume_accept for the local boolean instead

Comment thread test/mp/test/test.cpp Outdated
@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch 4 times, most recently from 57e7070 to 8511c68 Compare April 9, 2026 10:20
@enirox001

Copy link
Copy Markdown
Contributor Author

Thanks for the review @ryanofsky .

I addressed the cleanup points in the latest push:

  • _Serve is now explicit at all call sites
  • accept_pending is documented,
  • listener coverage now lives in a dedicated listen_tests.cpp file introduced in a separate commit before the local-limit extension.

I’m also planning to put together a draft Bitcoin Core PR using this API so the per-address approach can be evaluated downstream against the current global-limit direction.

@enirox001

enirox001 commented Apr 9, 2026

Copy link
Copy Markdown
Contributor Author

I put together the downstream draft using this API here bitcoin/bitcoin#35037

It uses per--ipcbind max-connections=<n> options, threads the parsed per-address limit into ListenConnections(), and adds downstream coverage for the local-limit behavior.

@enirox001

Copy link
Copy Markdown
Contributor Author

This PR is now ready for review

@xyzconstant xyzconstant left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 8511c68

Reviewed each commit separately, compiled and ran tests. The changes look good to me, only left a couple of inline nits noting a compilation error + failing tests in the first commit.

}
});
FooImplementation foo;
ListenConnections<messages::FooInterface>(loop, listener.release(), foo, max_connections);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This will fail to compile mptest at commit 8c47a3a since the 4th parameter max_connections is not introduced in ListenConnections's signature until next commit. I'd fix the call for this commit and then adjust the setup accordingly in 8511c68.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks

KJ_EXPECT(client->client->add(1, 2) == 3);
}

KJ_TEST("ListenConnections enforces a local connection limit")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Unlike in ListenConnections's call issue (compilation error), this will still fail for this commit (8c47a3a). Following ryanosfky's reasoning (#269 (comment)) I believe this suite would be better introduced in 8511c68 so the test lands with the feature it exercises.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@DrahtBot DrahtBot requested a review from ryanofsky May 13, 2026 03:15
@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch from 8511c68 to b36e98b Compare May 14, 2026 08:43
@enirox001

Copy link
Copy Markdown
Contributor Author

Thanks for the review @xyzconstant . Fixed both commit-structure issues: the first test commit now only adds baseline ListenConnections() coverage using the existing 3-argument API, and the max_connections setup/test now lands with the feature commit that introduces the new parameter

@enirox001

enirox001 commented May 14, 2026

Copy link
Copy Markdown
Contributor Author

Also tightened the capped listener behavior. It now stops posting accepts once the limit is reached, and keeps capped pending accepts alive so later clients can connect after an idle gap, including before the cap has been reached.

Added coverage for the reconnect cases as well

@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch from b36e98b to 19e1386 Compare May 14, 2026 08:51
Comment thread include/mp/proxy-io.h Outdated
[&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable {
state->accept_pending = false;
_ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream));
if (_ListenAtCapacity(*state)) return;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "proxy: add local connection limit to ListenConnections" (19e1386)

Not sure if I'm missing something but this check here seems redundant with the same _ListenAtCapacity check at the start of _Listen (line 889).

I tested commenting this line out and left the upper-level check (and vice-versa) and the tests passed with no issues. I'd suggest dropping any of these duplicates.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks. Dropped the lower _ListenAtCapacity() check since _Listen() already handles the capacity check before posting another accept.

@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch from 19e1386 to b0207dd Compare May 19, 2026 09:05
Comment thread include/mp/proxy-io.h Outdated
[&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable {
state->accept_pending = false;
_ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream));
_Listen<InitInterface>(loop, init, state);

@xyzconstant xyzconstant May 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the accept_pending check in place, this _Listen call will never be reached.

NOTE: if you apply this patch here (comment), then it will be needed here because we can't rely on the second _Listen call in _ServeAccepted which is executed conditionally after active_connections reached the cap.

@xyzconstant

Copy link
Copy Markdown
Contributor

Thanks for the update @enirox001!

I've been playing around with the PR code more throughly this time and have a different take on accept_pending now, so left a few comments.

Overall the code is factually correct and works as expected. And I think it could be merged (despite my latest thoughts on accept_pending). So I'll just re-ACK b0207dd

ryanofsky added a commit that referenced this pull request Jul 5, 2026
d499830 refactor: rename EventLoop::m_num_clients to m_num_refs (Sjors Provoost)

Pull request description:

  Rename the variable and document what it counts, so the `loop()` exit condition in `EventLoop::done()` is not misread as "exit when no clients are connected".

  I found myself confused by this while reviewing #269.

ACKs for top commit:
  ViniciusCestarii:
    ACK d499830, I agree this name is clearer.
  ryanofsky:
    Code review ACK d499830. Thanks for the rename and comment!

Tree-SHA512: cbef8bf61edc240bab65c5da1cef1572fef51d82c4be2f5ebac9186e3454e792481695a76c40eec844535b5b13a6f18022078c67e693ba9884d33823d6bb0bf4
@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch 5 times, most recently from edab7df to dcf9c38 Compare July 6, 2026 13:55
@enirox001

Copy link
Copy Markdown
Contributor Author

Thanks for all the reviews so far @Eunovo @Sjors @ryanofsky . Addressed all the comments in the latest commits.

@Eunovo Eunovo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK dcf9c38.


int release()
{
int fd = m_fd;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ef35679:

Not necessarily important because this is test code, but you can assert m_fd >= 0 before releasing here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, seems good to have.

Comment thread test/mp/test/listen_tests.cpp Outdated

auto client2 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());

// Without this sync, ConnectedCount() == 2 might pass even if

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dcf9c38:

I think you meant // Without this sync, ConnectedCount() == 1 might pass even if

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken, have renamed this

Comment thread test/mp/test/listen_tests.cpp Outdated

// Without this sync, ConnectedCount() == 2 might pass even if
// max_connections was not enforced because the event loop has not accepted
// client3 yet.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dcf9c38:

// client2 yet

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken thanks.

@DrahtBot DrahtBot requested a review from ryanofsky July 7, 2026 14:51

@ryanofsky ryanofsky left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK dcf9c38.

This looks good in it's current form and could be merged. Let me know if you'd prefer that or want to make another round of updates. I'm also planning to open a new bitcoin core PR bumping the subtree and incorporating this change after this PR is merged

Comment thread include/mp/proxy-io.h Outdated
//! Hook called on the worker thread just before returning results.
std::function<void()> testing_hook_async_request_done;

//! Hook called on the server thread when the client has connected.

@ryanofsky ryanofsky Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "test: add dedicated ListenConnections coverage" (ef35679)

Would change "server thread" to "event loop thread". Saying server thread is a little confusing because typically there's only one event loop and it processes all events on a single thread, not distinguishing between client and server events.

Would also change "the client" to "a client" since there may be more than one.

Same suggestion also applies to next commit adding disconnect hook.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, this does makes things clearer, the previous comment here does infer to having multiple server threads and a single client. Both of which are not correct.

I have applied the suggested changes.

Comment thread test/mp/test/listen_tests.cpp Outdated
public:
UnixListener()
{
char dir_template[] = "/tmp/mptest-listener-XXXXXX";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "test: add dedicated ListenConnections coverage" (ef35679)

Hardcoding /tmp is probably fine because this is just creating a socket, but it would be a little better to respect TMPDIR variable, maybe using std::filesystem::temp_directory_path() iike bitcoin tests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I changed this to use std::filesystem::temp_directory_path(), similar to the Bitcoin tests.

I also noticed that bitcoin core uses its own fs header, which is a more elaborate implementation for safer windows path handling. We don’t have that in libmultiprocess, and since Windows will be supported soon, this might also be a good place to look at using the libkj IO helpers in a follow-up

Comment thread test/mp/test/listen_tests.cpp Outdated
if (log.level == mp::Log::Raise) throw std::runtime_error(log.message);
});
loop.testing_hook_connected = [&] {
std::lock_guard<std::mutex> lock(counter_mutex);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "test: add dedicated ListenConnections coverage" (ef35679)

Would be a little better to use Mutex and Lock classes from util.h since they have thread safety annotations.

Same comment applies to new locks and mutexes added in the next commit.

(As possible followup it might be nice to have a linter that disallows non-annotated classes by default)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Switched the test counters to use the project Mutex and Lock wrappers instead of std::mutex and standard lock guards, and applied the same change to the locks added in the next commit.

Comment thread test/mp/test/listen_tests.cpp Outdated
void WaitForConnectedCount(size_t expected_count)
{
std::unique_lock<std::mutex> lock(counter_mutex);
const auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(5);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "test: add dedicated ListenConnections coverage" (ef35679)

Unexplained hardcoded timeouts in tests like this make code more difficult to understand and maintain. Would suggest defining constexpr auto FAILURE_TIMEOUT{30s}; similar to spawn_tests. The constant can then be used here and elsewhere for similar operations that are expected to complete quickly but will cause the test to fail if they do not.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted, having these hardoced timeouts are detrimental to clarity and maintenace. Added a named FAILURE_TIMEOUT constant and reused it for the wait operations, matching the pattern in spawn_tests.

Comment thread include/mp/proxy-io.h Outdated

template <typename InitInterface, typename InitImpl>
void _Listen(EventLoop& loop, kj::Own<kj::ConnectionReceiver>&& listener, InitImpl& init)
void Listener::listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<Listener>& self)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "test: add dedicated ListenConnections coverage" (ef35679)

I think it's a little confusing and creates an opportunity for bugs that a non-static method is taking a self parameter, and could potentially be called with two different Listener instances.

Also, if we want to start returning Listener objects from ListenConnections to applications to stop listening, it would be confusing for this method to be part of the public interface, since applications should never call it.

For both of these reasons would suggest dropping the Listen::listener method, and just keeping the previous _Listen function which is only meant to be used internally and not called by applications. This would also be a code simplification

Suggested change:

diff

--- a/include/mp/proxy-io.h
+++ b/include/mp/proxy-io.h
@@ -870,32 +870,27 @@ struct Listener
         return m_max_connections && m_active_connections >= *m_max_connections;
     }
 
-    //! Handle incoming connections by calling _Serve, to create ProxyServer
-    //! objects and forward requests to the init object.
-    template <typename InitInterface, typename InitImpl>
-    void listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<Listener>& self);
-
     kj::Own<kj::ConnectionReceiver> m_receiver;
     std::optional<size_t> m_max_connections;
     size_t m_active_connections{0};
 };
 
 template <typename InitInterface, typename InitImpl>
-void Listener::listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<Listener>& self)
+void _Listen(const std::shared_ptr<Listener>& listener, EventLoop& loop, InitImpl& init)
 {
-    if (atCapacity()) return;
+    if (listener->atCapacity()) return;
 
-    auto* receiver = m_receiver.get();
+    auto* receiver = listener->m_receiver.get();
     loop.m_task_set->add(receiver->accept().then(
-        [&loop, &init, self](kj::Own<kj::AsyncIoStream>&& stream) {
-            ++self->m_active_connections;
-            _Serve<InitInterface>(loop, kj::mv(stream), init, [&loop, &init, self] {
-                const bool resume_accept{self->atCapacity()};
-                assert(self->m_active_connections > 0);
-                --self->m_active_connections;
-                if (resume_accept) self->listen<InitInterface>(loop, init, self);
+        [&loop, &init, listener](kj::Own<kj::AsyncIoStream>&& stream) {
+            ++listener->m_active_connections;
+            _Serve<InitInterface>(loop, kj::mv(stream), init, [&loop, &init, listener] {
+                const bool resume_accept{listener->atCapacity()};
+                assert(listener->m_active_connections > 0);
+                --listener->m_active_connections;
+                if (resume_accept) _Listen<InitInterface>(listener, loop, init);
             });
-            self->listen<InitInterface>(loop, init, self);
+            _Listen<InitInterface>(listener, loop, init);
         }));
 }
 
@@ -920,7 +915,7 @@ void ListenConnections(EventLoop& loop, int fd, InitImpl& init, std::optional<si
         auto listener{std::make_shared<Listener>(
             loop.m_io_context.lowLevelProvider->wrapListenSocketFd(fd, kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP),
             max_connections)};
-        listener->listen<InitInterface>(loop, init, listener);
+        _Listen<InitInterface>(listener, loop, init);
     });
 }
 

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, changed this back to an internal _Listen() helper that takes the std::shared_ptr<Listener> directly

I had originally made this a Listener::listen() method because the async callbacks needed to capture a shared_ptr<Listener> to keep the listener alive, but I was probably thinking about the ownership problem in the wrong shape.

This approach keeps the same lifetime behavior without the confusing memberself parameter.

Comment thread test/mp/test/listen_tests.cpp Outdated
class ListenSetup
{
public:
ListenSetup()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "test: add dedicated ListenConnections coverage" (ef35679)

May want to declare this explicit now since it becomes explicit anyway in the next commit

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, marked as explicit in the test: add dedicated ListenConnections coverage commit

Comment thread doc/versions.md
## v12
- Current unstable version.

## [v11.0](https://github.com/bitcoin-core/libmultiprocess/commits/v11.0)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "doc/version: Bump version 11 > 12" (2f7f1bf)

These updates look right but might better to use the more complete list of changes from bitcoin/bitcoin#35661. Suggested diff is below, but also feel free to keep current test or just replace the list with a TBD comment.

diff

--- a/doc/versions.md
+++ b/doc/versions.md
@@ -14,11 +14,17 @@ include.
   and resume accepting after existing connections disconnect.
 
 ## [v11.0](https://github.com/bitcoin-core/libmultiprocess/commits/v11.0)
-- Tolerates unexpected exceptions in event loop `post()` callbacks.
-- Tolerates exceptions from remote destroy during cleanup in `ProxyClient`.
-- Supports primitive `std::optional` struct fields in the code generator (`mpgen`).
-- Adds `TypeName()` and improves debug log coverage for Proxy object lifecycle.
-- Updates build compatibility with recent Nix and CMake versions.
+- Adds `makePool` method on `ThreadMap` to support thread pool routing, allowing requests without a specific client thread to be dispatched to a pool using a shortest-queue strategy ([#283](https://github.com/bitcoin-core/libmultiprocess/pull/283)).
+- Adds `std::unordered_set` support, a `BuildList` helper, and a `ReadList` helper to reduce duplication in list build and read handlers ([#277](https://github.com/bitcoin-core/libmultiprocess/pull/277), [#285](https://github.com/bitcoin-core/libmultiprocess/pull/285)).
+- Adds support for translating C++ `std::optional<T>` struct fields to pairs of `T` + `hasT :Bool` Cap'n Proto struct fields, allowing unset optional primitive fields to be represented ([#243](https://github.com/bitcoin-core/libmultiprocess/pull/243)).
+- Produces more readable log output for Proxy object lifecycle events and IPC server-side failures ([#218](https://github.com/bitcoin-core/libmultiprocess/pull/218)).
+- Handles exceptions thrown by `destroy` methods by logging instead of aborting ([#273](https://github.com/bitcoin-core/libmultiprocess/pull/273)). This can prevent server crashes when non-libmultiprocess clients disconnect without destroying objects, in the case where a server object owns client objects and the server destructor tries to call the disconnected client to free them ([#219](https://github.com/bitcoin-core/libmultiprocess/issues/219)).
+- Handles unexpected exceptions thrown by callbacks (that should never happen) by logging errors instead of deadlocking ([#260](https://github.com/bitcoin-core/libmultiprocess/pull/260)).
+- Fixes a rare mptest hang on musl builds caused by a lost wakeup bug in `Waiter` ([#295](https://github.com/bitcoin-core/libmultiprocess/pull/295)).
+- Fixes a race condition in a log print detected by TSan ([#286](https://github.com/bitcoin-core/libmultiprocess/pull/286)).
+- Build improvements: makes `target_capnp_sources` work correctly when libmultiprocess is used as a CMake subproject ([#289](https://github.com/bitcoin-core/libmultiprocess/pull/289)), adds `mp_headers` target for better lint tool support ([#291](https://github.com/bitcoin-core/libmultiprocess/pull/291)), and fixes compatibility with recent Nix and CMake 4.0 ([#238](https://github.com/bitcoin-core/libmultiprocess/pull/238)).
+- Test, CI, documentation, and minor code improvements: design document corrections ([#278](https://github.com/bitcoin-core/libmultiprocess/pull/278)), field constant comments ([#279](https://github.com/bitcoin-core/libmultiprocess/pull/279)), clang-tidy fix ([#292](https://github.com/bitcoin-core/libmultiprocess/pull/292)), new smoke test for double-precision float values ([#294](https://github.com/bitcoin-core/libmultiprocess/pull/294)), new test for recursive async IPC calls ([#301](https://github.com/bitcoin-core/libmultiprocess/pull/301)), removal of libevent from Core CI builds ([#299](https://github.com/bitcoin-core/libmultiprocess/pull/299)), and rename of `EventLoop::m_num_clients` to `m_num_refs` ([#302](https://github.com/bitcoin-core/libmultiprocess/pull/302)).
+- Used in Bitcoin Core master branch, pulled in by [#35661](https://github.com/bitcoin/bitcoin/pull/35661).
 
 ## [v10.0](https://github.com/bitcoin-core/libmultiprocess/commits/v10.0)
 - Increases spawn test timeout to avoid spurious failures.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I replaced the shorter v11 list with the more complete notes from bitcoin/bitcoin#35661. That seems better than leaving a partial summary here

server.WaitForDisconnectedCount(2);

auto client3 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
server.WaitForConnectedCount(3);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "proxy: add local connection limit to ListenConnections()" (dcf9c38)

I think it would make sense before each WaitFor{Connected,Disconnected}Count call to have a KJ_EXPECT call checking the previous value. Should make the test stronger and also clearer and easier to debug if something is wrong

diff

--- a/test/mp/test/listen_tests.cpp
+++ b/test/mp/test/listen_tests.cpp
@@ -168,6 +168,12 @@ public:
         return connected_count;
     }
 
+    size_t DisconnectedCount()
+    {
+        std::lock_guard<std::mutex> lock(counter_mutex);
+        return disconnected_count;
+    }
+
     void WaitForConnectedCount(size_t expected_count)
     {
         std::unique_lock<std::mutex> lock(counter_mutex);
@@ -203,8 +209,8 @@ public:
 KJ_TEST("ListenConnections accepts incoming connections")
 {
     ListenSetup server;
+    KJ_EXPECT(server.ConnectedCount() == 0);
     auto client = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
-
     server.WaitForConnectedCount(1);
     KJ_EXPECT(client->client->add(1, 2) == 3);
 }
@@ -217,29 +223,34 @@ KJ_TEST("ListenConnections enforces a local connection limit")
 
     ListenSetup server(/*max_connections=*/1);
 
+    KJ_EXPECT(server.ConnectedCount() == 0);
     auto client1 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
     server.WaitForConnectedCount(1);
+
     KJ_EXPECT(client1->client->add(1, 2) == 3);
 
     auto client2 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
-
     // Without this sync, ConnectedCount() == 2 might pass even if
     // max_connections was not enforced because the event loop has not accepted
     // client3 yet.
     (**server.m_loop_ref).sync([] {});
-    KJ_EXPECT(server.ConnectedCount() == 1);
 
+    KJ_EXPECT(server.ConnectedCount() == 1);
+    KJ_EXPECT(server.DisconnectedCount() == 0);
     client1.reset();
     server.WaitForDisconnectedCount(1);
     server.WaitForConnectedCount(2);
 
     KJ_EXPECT(client2->client->add(2, 3) == 5);
 
+    KJ_EXPECT(server.DisconnectedCount() == 1);
     client2.reset();
     server.WaitForDisconnectedCount(2);
 
+    KJ_EXPECT(server.ConnectedCount() == 2);
     auto client3 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
     server.WaitForConnectedCount(3);
+
     KJ_EXPECT(client3->client->add(3, 4) == 7);
 }
 
@@ -250,22 +261,22 @@ KJ_TEST("ListenConnections accepts multiple connections")
     
     ListenSetup server(/*max_connections=*/2);
 
+    KJ_EXPECT(server.ConnectedCount() == 0);
     auto client1 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
     auto client2 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
-
     server.WaitForConnectedCount(2);
 
     KJ_EXPECT(client1->client->add(1, 2) == 3);
     KJ_EXPECT(client2->client->add(2, 3) == 5);
 
     auto client3 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
-
     // Without this sync, ConnectedCount() == 2 might pass even if
     // max_connections was not enforced because the event loop has not accepted
     // client3 yet.
     (**server.m_loop_ref).sync([] {});
-    KJ_EXPECT(server.ConnectedCount() == 2);
 
+    KJ_EXPECT(server.ConnectedCount() == 2);
+    KJ_EXPECT(server.DisconnectedCount() == 0);
     client1.reset();
     server.WaitForDisconnectedCount(1);
     server.WaitForConnectedCount(3);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. added this helper as well as included in the relevant tests

@enirox001

Copy link
Copy Markdown
Contributor Author

This looks good in it's current form and could be merged. Let me know if you'd prefer that or want to make another round of updates

Thanks for giving this a look, i intend to make another round of updates. Should be able to have that done soon

This bumps the major version to 12 because upcoming commits
introduce a non-trivial feature by adding a local max-connections
parameter to the ListenConnections() method

This also records release notes for v11 in doc/version.md. These notes
are unrelated to this PR and describe changes that will be tagges before
this PR
@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch 3 times, most recently from 683bee9 to e4dad41 Compare July 8, 2026 11:47
@enirox001

Copy link
Copy Markdown
Contributor Author

Thanks for the reviews @ryanofsky @Eunovo. Addressed them in the latest commits

I'm also planning to open a new bitcoin core PR bumping the subtree and incorporating this change after this PR is merged

This would be helpful to bitcoin/bitcoin#35037 as well as greatly simplify the work needed for the PR. Looking forward to reviewing this

@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch 2 times, most recently from 95aaf9e to f9ef92e Compare July 8, 2026 12:13
Comment thread test/mp/test/listen_tests.cpp Outdated
Add a separate listen_tests.cpp file with reusable UnixListener, ClientSetup
and ListenSetup helpers for exercising ListenConnections() with real Unix
domain sockets.

The new test covers the baseline behavior that ListenConnections() accepts an
incoming connection and serves requests over it. Keeping this coverage separate
from the existing general proxy tests makes the socket listener setup easier to
review and provides a clearer place to extend listener-specific behavior in
follow-up commits.

@ryanofsky ryanofsky left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK f9ef92e. Thanks for the updates!Feel free to ignore my comment about whitespace. I only posted it since I noticed CI was failing and thought there would have to be another push

@DrahtBot DrahtBot requested a review from Eunovo July 8, 2026 12:21
@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch from f9ef92e to e63835c Compare July 8, 2026 12:21
Add an optional max_connections parameter to ListenConnections() and track
the limit with listener-local active connection state, so accepting pauses
at capacity and resumes after a disconnect.

Update listener tests for cap enforcement, resume behavior, and multiple
active connections.
@enirox001 enirox001 force-pushed the 04-26-ipc-local-connection-limit branch from e63835c to 39a10ce Compare July 8, 2026 12:30
@enirox001

Copy link
Copy Markdown
Contributor Author

Thanks for the updates!Feel free to ignore my comment about whitespace. I only posted it since I noticed CI was failing and thought there would have to be another push

Addressed these in the recent change. Faced some IWYU errors after the comment above. So had to resolve them anyway. Thanks

@ryanofsky ryanofsky left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 39a10ce

Just include and whitespace changes since last review

@ryanofsky ryanofsky merged commit 28e0565 into bitcoin-core:master Jul 8, 2026
13 checks passed
@xyzconstant

Copy link
Copy Markdown
Contributor

Post-merge re-ACK 39a10ce

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants