proxy: add local connection limit to ListenConnections#269
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline and AI policy for information on the review process.
If your review is incorrectly listed, please copy-paste ConflictsNo conflicts as of last run. |
3ef8e5c to
84ed607
Compare
ryanofsky
left a comment
There was a problem hiding this comment.
Approach ACK 84ed607. Implementation of local connection limit here looks almost exactly like I would have expected.
I do think it would be helpful to see a draft PR in the bitcoin repo using this API (it should be fine to make libmultiprocess changes there and let the lint CI job fail so you don't need to mess with subtrees) because the approach in bitcoin/bitcoin#34978 of adding a global connection limit option isn't exactly compatible with the implementation here of implementing a per-address connection limit.
I'd personally prefer using per-address limits over introducing a global limit but both approaches seem reasonable
| kj::Own<kj::ConnectionReceiver> listener; | ||
| std::optional<size_t> max_connections; | ||
| size_t active_connections{0}; | ||
| bool accept_pending{false}; |
There was a problem hiding this comment.
In commit "proxy: add local connection limit to ListenConnections" (84ed607)
Curious if this accept_pending variable is actually necessary or if code could just compare active_connections and max_connections when deciding whether to listen. Would prefer to avoid redundancy in the state representation if possible even if makes individual checks more a little more verbose.
If accept_pending really is necessary would be a good to have a short comment about why.
There was a problem hiding this comment.
I kept accept_pending, but added a short comment explaining why it is needed here. active_connections only counts accepted connections, so without a separate flag nested _Listen() calls could post multiple pending accept() calls before active_connections is incremented.
There was a problem hiding this comment.
FWIW I tested the code without accept_pending field and hence its check at _Listen removed, the tests passed with no issues. Then, asked Claude to generate a test that exercises it and produced this:
diff --git a/test/mp/test/listen_tests.cpp b/test/mp/test/listen_tests.cpp
index b367938..78298c7 100644
--- a/test/mp/test/listen_tests.cpp
+++ b/test/mp/test/listen_tests.cpp
@@ -24,8 +24,10 @@
#include <string>
#include <sys/socket.h>
#include <sys/un.h>
+#include <kj/exception.h>
#include <thread>
#include <unistd.h>
+#include <vector>
namespace mp {
namespace test {
@@ -112,11 +114,39 @@ public:
std::thread thread;
};
+//! kj::ExceptionCallback that captures KJ_LOG output into an external sink.
+//! Must be instantiated on the thread whose KJ logs you want to capture; it
+//! installs itself onto that thread's ExceptionCallback stack via its base
+//! constructor and removes itself in the destructor.
+class CaptureLogCallback : public kj::ExceptionCallback
+{
+public:
+ CaptureLogCallback(std::mutex& mu, std::string& sink) : m_mu(mu), m_sink(sink) {}
+
+ void logMessage(kj::LogSeverity severity, const char* file, int line, int contextDepth,
+ kj::String&& text) override
+ {
+ {
+ std::lock_guard<std::mutex> lock(m_mu);
+ m_sink.append(text.cStr(), text.size());
+ m_sink.push_back('\n');
+ }
+ // Still let the default callback emit to stderr so test debug output
+ // isn't silenced for other observers.
+ kj::ExceptionCallback::logMessage(severity, file, line, contextDepth, kj::mv(text));
+ }
+
+private:
+ std::mutex& m_mu;
+ std::string& m_sink;
+};
+
class ListenSetup
{
public:
explicit ListenSetup(std::optional<size_t> max_connections = std::nullopt)
: capped_listener(max_connections.has_value()), thread([this, max_connections] {
+ CaptureLogCallback log_capture(captured_log_mutex, captured_log);
EventLoop loop("mptest-server", [this](mp::LogMessage log) {
if (log.level == mp::Log::Raise) throw std::runtime_error(log.message);
if (log.message.find("IPC server: socket connected.") != std::string::npos) {
@@ -144,6 +174,18 @@ public:
~ListenSetup()
{
+ forceShutdown();
+ thread.join();
+ }
+
+ //! Synchronously tear down the event loop's task set so any pending accept
+ //! promises are destroyed now (rather than when the destructor runs later).
+ //! This makes it possible to assert on captured KJ log output before the
+ //! ListenSetup goes out of scope. Idempotent.
+ void forceShutdown()
+ {
+ if (shutdown_done) return;
+ shutdown_done = true;
if (capped_listener) {
EventLoop* loop;
{
@@ -152,7 +194,6 @@ public:
}
if (loop) loop->sync([&] { loop->m_task_set.reset(); });
}
- thread.join();
}
size_t ConnectedCount()
@@ -184,11 +225,15 @@ public:
UnixListener listener;
std::promise<void> ready_promise;
bool capped_listener{false};
+ bool shutdown_done{false};
std::mutex counter_mutex;
std::condition_variable counter_cv;
EventLoop* event_loop{nullptr};
size_t connected_count{0};
size_t disconnected_count{0};
+ //! KJ log output captured from the server thread via CaptureLogCallback.
+ std::mutex captured_log_mutex;
+ std::string captured_log;
std::thread thread;
};
@@ -245,6 +290,34 @@ KJ_TEST("ListenConnections keeps capped listeners alive before reaching the limi
KJ_EXPECT(client2->client->add(2, 3) == 5);
}
+// Without `accept_pending`, cascaded close handlers each post a duplicate
+// accept(). KJ silently serializes them so the cap isn't exceeded, but the
+// extra pending promises are destroyed at cleanup and logged as
+// "PromiseFulfiller was destroyed without fulfilling the promise."
+// This test fails when accept_pending is removed and passes when it's intact.
+KJ_TEST("ListenConnections does not leak accept promises during disconnect burst")
+{
+ constexpr size_t kCap = 2;
+ ListenSetup setup(/*max_connections=*/kCap);
+
+ std::vector<std::unique_ptr<ClientSetup>> filling;
+ filling.reserve(kCap);
+ for (size_t i = 0; i < kCap; ++i) {
+ filling.push_back(std::make_unique<ClientSetup>(setup.listener.Connect()));
+ }
+ setup.WaitForConnectedCount(kCap);
+
+ filling.clear();
+ setup.WaitForDisconnectedCount(kCap);
+
+ // Trigger m_task_set.reset() now so any leaked accept promises get destroyed
+ // before we read captured_log.
+ setup.forceShutdown();
+
+ std::lock_guard<std::mutex> lock(setup.captured_log_mutex);
+ KJ_EXPECT(setup.captured_log.find("PromiseFulfiller was destroyed") == std::string::npos);
+}
+
} // namespace
} // namespace test
} // namespace mpBasically what this does is install a kj::ExceptionCallback in the server thread to capture the log "PromiseFulfiller was destroyed" generated by KJ runtime if cascaded disconnects each call _Listen and post a fresh accept() promise. The test fails with accept_pending removed and pass with it back.
There was a problem hiding this comment.
IMO it's not so obvious why this field is needed here, this patch basically guarantee the same outcome and also pass the test generated by Claude above:
diff --git a/include/mp/proxy-io.h b/include/mp/proxy-io.h
index 78924c6..c002811 100644
--- a/include/mp/proxy-io.h
+++ b/include/mp/proxy-io.h
@@ -851,11 +851,6 @@ struct ListenState
kj::Own<kj::ConnectionReceiver> listener;
std::optional<size_t> max_connections;
size_t active_connections{0};
- //! Tracks whether accept() has already been posted. This is needed because
- //! active_connections only counts accepted connections, so without a
- //! separate flag, nested _Listen() calls could queue multiple pending
- //! accepts before active_connections increases.
- bool accept_pending{false};
};
template <typename InitInterface, typename InitImpl>
@@ -866,9 +861,12 @@ void _ServeAccepted(EventLoop& loop, InitImpl& init, const std::shared_ptr<Liste
{
++state->active_connections;
_Serve<InitInterface>(loop, kj::mv(stream), init, [&loop, &init, state] {
+ const bool was_at_cap = state->max_connections && state->active_connections == *state->max_connections;
assert(state->active_connections > 0);
--state->active_connections;
- _Listen<InitInterface>(loop, init, state);
+ if (was_at_cap) {
+ _Listen<InitInterface>(loop, init, state);
+ }
});
}
@@ -885,15 +883,12 @@ inline std::unique_ptr<EventLoopRef> _MakeCappedListenerRef(EventLoop& loop, con
template <typename InitInterface, typename InitImpl>
void _Listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<ListenState>& state)
{
- if (state->accept_pending) return;
if (_ListenAtCapacity(*state)) return;
- state->accept_pending = true;
auto* ptr = state->listener.get();
auto accept_ref{_MakeCappedListenerRef(loop, *state)};
loop.m_task_set->add(ptr->accept().then(
[&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable {
- state->accept_pending = false;
_ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream));
_Listen<InitInterface>(loop, init, state);
}));There was a problem hiding this comment.
I think this is a better invariant than tracking accept_pending.
There should already be one pending accept whenever the capped listener is below capacity, and disconnects only need to post a new accept when they transition the listener from full to below full, because that is the only state where nocaccept was pending.
So checking this before decrementing active_connections avoids the duplicate-accept case without adding another state variable.
Taken and decided to use resume_accept for the local boolean instead
57e7070 to
8511c68
Compare
|
Thanks for the review @ryanofsky . I addressed the cleanup points in the latest push:
I’m also planning to put together a draft Bitcoin Core PR using this API so the per-address approach can be evaluated downstream against the current global-limit direction. |
|
I put together the downstream draft using this API here bitcoin/bitcoin#35037 It uses per |
|
This PR is now ready for review |
xyzconstant
left a comment
There was a problem hiding this comment.
Code review ACK 8511c68
Reviewed each commit separately, compiled and ran tests. The changes look good to me, only left a couple of inline nits noting a compilation error + failing tests in the first commit.
| } | ||
| }); | ||
| FooImplementation foo; | ||
| ListenConnections<messages::FooInterface>(loop, listener.release(), foo, max_connections); |
| KJ_EXPECT(client->client->add(1, 2) == 3); | ||
| } | ||
|
|
||
| KJ_TEST("ListenConnections enforces a local connection limit") |
There was a problem hiding this comment.
nit: Unlike in ListenConnections's call issue (compilation error), this will still fail for this commit (8c47a3a). Following ryanosfky's reasoning (#269 (comment)) I believe this suite would be better introduced in 8511c68 so the test lands with the feature it exercises.
8511c68 to
b36e98b
Compare
|
Thanks for the review @xyzconstant . Fixed both commit-structure issues: the first test commit now only adds baseline |
|
Also tightened the capped listener behavior. It now stops posting accepts once the limit is reached, and keeps capped pending accepts alive so later clients can connect after an idle gap, including before the cap has been reached. Added coverage for the reconnect cases as well |
b36e98b to
19e1386
Compare
| [&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable { | ||
| state->accept_pending = false; | ||
| _ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream)); | ||
| if (_ListenAtCapacity(*state)) return; |
There was a problem hiding this comment.
In commit "proxy: add local connection limit to ListenConnections" (19e1386)
Not sure if I'm missing something but this check here seems redundant with the same _ListenAtCapacity check at the start of _Listen (line 889).
I tested commenting this line out and left the upper-level check (and vice-versa) and the tests passed with no issues. I'd suggest dropping any of these duplicates.
There was a problem hiding this comment.
Good catch, thanks. Dropped the lower _ListenAtCapacity() check since _Listen() already handles the capacity check before posting another accept.
19e1386 to
b0207dd
Compare
| [&loop, &init, state, accept_ref = std::move(accept_ref)](kj::Own<kj::AsyncIoStream>&& stream) mutable { | ||
| state->accept_pending = false; | ||
| _ServeAccepted<InitInterface>(loop, init, state, kj::mv(stream)); | ||
| _Listen<InitInterface>(loop, init, state); |
There was a problem hiding this comment.
With the accept_pending check in place, this _Listen call will never be reached.
NOTE: if you apply this patch here (comment), then it will be needed here because we can't rely on the second _Listen call in _ServeAccepted which is executed conditionally after active_connections reached the cap.
|
Thanks for the update @enirox001! I've been playing around with the PR code more throughly this time and have a different take on Overall the code is factually correct and works as expected. And I think it could be merged (despite my latest thoughts on |
d499830 refactor: rename EventLoop::m_num_clients to m_num_refs (Sjors Provoost) Pull request description: Rename the variable and document what it counts, so the `loop()` exit condition in `EventLoop::done()` is not misread as "exit when no clients are connected". I found myself confused by this while reviewing #269. ACKs for top commit: ViniciusCestarii: ACK d499830, I agree this name is clearer. ryanofsky: Code review ACK d499830. Thanks for the rename and comment! Tree-SHA512: cbef8bf61edc240bab65c5da1cef1572fef51d82c4be2f5ebac9186e3454e792481695a76c40eec844535b5b13a6f18022078c67e693ba9884d33823d6bb0bf4
edab7df to
dcf9c38
Compare
|
Thanks for all the reviews so far @Eunovo @Sjors @ryanofsky . Addressed all the comments in the latest commits. |
|
|
||
| int release() | ||
| { | ||
| int fd = m_fd; |
There was a problem hiding this comment.
Not necessarily important because this is test code, but you can assert m_fd >= 0 before releasing here.
There was a problem hiding this comment.
Added, seems good to have.
|
|
||
| auto client2 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket()); | ||
|
|
||
| // Without this sync, ConnectedCount() == 2 might pass even if |
There was a problem hiding this comment.
I think you meant // Without this sync, ConnectedCount() == 1 might pass even if
There was a problem hiding this comment.
Taken, have renamed this
|
|
||
| // Without this sync, ConnectedCount() == 2 might pass even if | ||
| // max_connections was not enforced because the event loop has not accepted | ||
| // client3 yet. |
There was a problem hiding this comment.
Code review ACK dcf9c38.
This looks good in it's current form and could be merged. Let me know if you'd prefer that or want to make another round of updates. I'm also planning to open a new bitcoin core PR bumping the subtree and incorporating this change after this PR is merged
| //! Hook called on the worker thread just before returning results. | ||
| std::function<void()> testing_hook_async_request_done; | ||
|
|
||
| //! Hook called on the server thread when the client has connected. |
There was a problem hiding this comment.
In commit "test: add dedicated ListenConnections coverage" (ef35679)
Would change "server thread" to "event loop thread". Saying server thread is a little confusing because typically there's only one event loop and it processes all events on a single thread, not distinguishing between client and server events.
Would also change "the client" to "a client" since there may be more than one.
Same suggestion also applies to next commit adding disconnect hook.
There was a problem hiding this comment.
I agree, this does makes things clearer, the previous comment here does infer to having multiple server threads and a single client. Both of which are not correct.
I have applied the suggested changes.
| public: | ||
| UnixListener() | ||
| { | ||
| char dir_template[] = "/tmp/mptest-listener-XXXXXX"; |
There was a problem hiding this comment.
In commit "test: add dedicated ListenConnections coverage" (ef35679)
Hardcoding /tmp is probably fine because this is just creating a socket, but it would be a little better to respect TMPDIR variable, maybe using std::filesystem::temp_directory_path() iike bitcoin tests
There was a problem hiding this comment.
Thanks, I changed this to use std::filesystem::temp_directory_path(), similar to the Bitcoin tests.
I also noticed that bitcoin core uses its own fs header, which is a more elaborate implementation for safer windows path handling. We don’t have that in libmultiprocess, and since Windows will be supported soon, this might also be a good place to look at using the libkj IO helpers in a follow-up
| if (log.level == mp::Log::Raise) throw std::runtime_error(log.message); | ||
| }); | ||
| loop.testing_hook_connected = [&] { | ||
| std::lock_guard<std::mutex> lock(counter_mutex); |
There was a problem hiding this comment.
In commit "test: add dedicated ListenConnections coverage" (ef35679)
Would be a little better to use Mutex and Lock classes from util.h since they have thread safety annotations.
Same comment applies to new locks and mutexes added in the next commit.
(As possible followup it might be nice to have a linter that disallows non-annotated classes by default)
There was a problem hiding this comment.
Done. Switched the test counters to use the project Mutex and Lock wrappers instead of std::mutex and standard lock guards, and applied the same change to the locks added in the next commit.
| void WaitForConnectedCount(size_t expected_count) | ||
| { | ||
| std::unique_lock<std::mutex> lock(counter_mutex); | ||
| const auto deadline = std::chrono::steady_clock::now() + std::chrono::seconds(5); |
There was a problem hiding this comment.
In commit "test: add dedicated ListenConnections coverage" (ef35679)
Unexplained hardcoded timeouts in tests like this make code more difficult to understand and maintain. Would suggest defining constexpr auto FAILURE_TIMEOUT{30s}; similar to spawn_tests. The constant can then be used here and elsewhere for similar operations that are expected to complete quickly but will cause the test to fail if they do not.
There was a problem hiding this comment.
Noted, having these hardoced timeouts are detrimental to clarity and maintenace. Added a named FAILURE_TIMEOUT constant and reused it for the wait operations, matching the pattern in spawn_tests.
|
|
||
| template <typename InitInterface, typename InitImpl> | ||
| void _Listen(EventLoop& loop, kj::Own<kj::ConnectionReceiver>&& listener, InitImpl& init) | ||
| void Listener::listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<Listener>& self) |
There was a problem hiding this comment.
In commit "test: add dedicated ListenConnections coverage" (ef35679)
I think it's a little confusing and creates an opportunity for bugs that a non-static method is taking a self parameter, and could potentially be called with two different Listener instances.
Also, if we want to start returning Listener objects from ListenConnections to applications to stop listening, it would be confusing for this method to be part of the public interface, since applications should never call it.
For both of these reasons would suggest dropping the Listen::listener method, and just keeping the previous _Listen function which is only meant to be used internally and not called by applications. This would also be a code simplification
Suggested change:
diff
--- a/include/mp/proxy-io.h
+++ b/include/mp/proxy-io.h
@@ -870,32 +870,27 @@ struct Listener
return m_max_connections && m_active_connections >= *m_max_connections;
}
- //! Handle incoming connections by calling _Serve, to create ProxyServer
- //! objects and forward requests to the init object.
- template <typename InitInterface, typename InitImpl>
- void listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<Listener>& self);
-
kj::Own<kj::ConnectionReceiver> m_receiver;
std::optional<size_t> m_max_connections;
size_t m_active_connections{0};
};
template <typename InitInterface, typename InitImpl>
-void Listener::listen(EventLoop& loop, InitImpl& init, const std::shared_ptr<Listener>& self)
+void _Listen(const std::shared_ptr<Listener>& listener, EventLoop& loop, InitImpl& init)
{
- if (atCapacity()) return;
+ if (listener->atCapacity()) return;
- auto* receiver = m_receiver.get();
+ auto* receiver = listener->m_receiver.get();
loop.m_task_set->add(receiver->accept().then(
- [&loop, &init, self](kj::Own<kj::AsyncIoStream>&& stream) {
- ++self->m_active_connections;
- _Serve<InitInterface>(loop, kj::mv(stream), init, [&loop, &init, self] {
- const bool resume_accept{self->atCapacity()};
- assert(self->m_active_connections > 0);
- --self->m_active_connections;
- if (resume_accept) self->listen<InitInterface>(loop, init, self);
+ [&loop, &init, listener](kj::Own<kj::AsyncIoStream>&& stream) {
+ ++listener->m_active_connections;
+ _Serve<InitInterface>(loop, kj::mv(stream), init, [&loop, &init, listener] {
+ const bool resume_accept{listener->atCapacity()};
+ assert(listener->m_active_connections > 0);
+ --listener->m_active_connections;
+ if (resume_accept) _Listen<InitInterface>(listener, loop, init);
});
- self->listen<InitInterface>(loop, init, self);
+ _Listen<InitInterface>(listener, loop, init);
}));
}
@@ -920,7 +915,7 @@ void ListenConnections(EventLoop& loop, int fd, InitImpl& init, std::optional<si
auto listener{std::make_shared<Listener>(
loop.m_io_context.lowLevelProvider->wrapListenSocketFd(fd, kj::LowLevelAsyncIoProvider::TAKE_OWNERSHIP),
max_connections)};
- listener->listen<InitInterface>(loop, init, listener);
+ _Listen<InitInterface>(listener, loop, init);
});
}
There was a problem hiding this comment.
Done, changed this back to an internal _Listen() helper that takes the std::shared_ptr<Listener> directly
I had originally made this a Listener::listen() method because the async callbacks needed to capture a shared_ptr<Listener> to keep the listener alive, but I was probably thinking about the ownership problem in the wrong shape.
This approach keeps the same lifetime behavior without the confusing memberself parameter.
| class ListenSetup | ||
| { | ||
| public: | ||
| ListenSetup() |
There was a problem hiding this comment.
In commit "test: add dedicated ListenConnections coverage" (ef35679)
May want to declare this explicit now since it becomes explicit anyway in the next commit
There was a problem hiding this comment.
Done, marked as explicit in the test: add dedicated ListenConnections coverage commit
| ## v12 | ||
| - Current unstable version. | ||
|
|
||
| ## [v11.0](https://github.com/bitcoin-core/libmultiprocess/commits/v11.0) |
There was a problem hiding this comment.
In commit "doc/version: Bump version 11 > 12" (2f7f1bf)
These updates look right but might better to use the more complete list of changes from bitcoin/bitcoin#35661. Suggested diff is below, but also feel free to keep current test or just replace the list with a TBD comment.
diff
--- a/doc/versions.md
+++ b/doc/versions.md
@@ -14,11 +14,17 @@ include.
and resume accepting after existing connections disconnect.
## [v11.0](https://github.com/bitcoin-core/libmultiprocess/commits/v11.0)
-- Tolerates unexpected exceptions in event loop `post()` callbacks.
-- Tolerates exceptions from remote destroy during cleanup in `ProxyClient`.
-- Supports primitive `std::optional` struct fields in the code generator (`mpgen`).
-- Adds `TypeName()` and improves debug log coverage for Proxy object lifecycle.
-- Updates build compatibility with recent Nix and CMake versions.
+- Adds `makePool` method on `ThreadMap` to support thread pool routing, allowing requests without a specific client thread to be dispatched to a pool using a shortest-queue strategy ([#283](https://github.com/bitcoin-core/libmultiprocess/pull/283)).
+- Adds `std::unordered_set` support, a `BuildList` helper, and a `ReadList` helper to reduce duplication in list build and read handlers ([#277](https://github.com/bitcoin-core/libmultiprocess/pull/277), [#285](https://github.com/bitcoin-core/libmultiprocess/pull/285)).
+- Adds support for translating C++ `std::optional<T>` struct fields to pairs of `T` + `hasT :Bool` Cap'n Proto struct fields, allowing unset optional primitive fields to be represented ([#243](https://github.com/bitcoin-core/libmultiprocess/pull/243)).
+- Produces more readable log output for Proxy object lifecycle events and IPC server-side failures ([#218](https://github.com/bitcoin-core/libmultiprocess/pull/218)).
+- Handles exceptions thrown by `destroy` methods by logging instead of aborting ([#273](https://github.com/bitcoin-core/libmultiprocess/pull/273)). This can prevent server crashes when non-libmultiprocess clients disconnect without destroying objects, in the case where a server object owns client objects and the server destructor tries to call the disconnected client to free them ([#219](https://github.com/bitcoin-core/libmultiprocess/issues/219)).
+- Handles unexpected exceptions thrown by callbacks (that should never happen) by logging errors instead of deadlocking ([#260](https://github.com/bitcoin-core/libmultiprocess/pull/260)).
+- Fixes a rare mptest hang on musl builds caused by a lost wakeup bug in `Waiter` ([#295](https://github.com/bitcoin-core/libmultiprocess/pull/295)).
+- Fixes a race condition in a log print detected by TSan ([#286](https://github.com/bitcoin-core/libmultiprocess/pull/286)).
+- Build improvements: makes `target_capnp_sources` work correctly when libmultiprocess is used as a CMake subproject ([#289](https://github.com/bitcoin-core/libmultiprocess/pull/289)), adds `mp_headers` target for better lint tool support ([#291](https://github.com/bitcoin-core/libmultiprocess/pull/291)), and fixes compatibility with recent Nix and CMake 4.0 ([#238](https://github.com/bitcoin-core/libmultiprocess/pull/238)).
+- Test, CI, documentation, and minor code improvements: design document corrections ([#278](https://github.com/bitcoin-core/libmultiprocess/pull/278)), field constant comments ([#279](https://github.com/bitcoin-core/libmultiprocess/pull/279)), clang-tidy fix ([#292](https://github.com/bitcoin-core/libmultiprocess/pull/292)), new smoke test for double-precision float values ([#294](https://github.com/bitcoin-core/libmultiprocess/pull/294)), new test for recursive async IPC calls ([#301](https://github.com/bitcoin-core/libmultiprocess/pull/301)), removal of libevent from Core CI builds ([#299](https://github.com/bitcoin-core/libmultiprocess/pull/299)), and rename of `EventLoop::m_num_clients` to `m_num_refs` ([#302](https://github.com/bitcoin-core/libmultiprocess/pull/302)).
+- Used in Bitcoin Core master branch, pulled in by [#35661](https://github.com/bitcoin/bitcoin/pull/35661).
## [v10.0](https://github.com/bitcoin-core/libmultiprocess/commits/v10.0)
- Increases spawn test timeout to avoid spurious failures.There was a problem hiding this comment.
Thanks, I replaced the shorter v11 list with the more complete notes from bitcoin/bitcoin#35661. That seems better than leaving a partial summary here
| server.WaitForDisconnectedCount(2); | ||
|
|
||
| auto client3 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket()); | ||
| server.WaitForConnectedCount(3); |
There was a problem hiding this comment.
In commit "proxy: add local connection limit to ListenConnections()" (dcf9c38)
I think it would make sense before each WaitFor{Connected,Disconnected}Count call to have a KJ_EXPECT call checking the previous value. Should make the test stronger and also clearer and easier to debug if something is wrong
diff
--- a/test/mp/test/listen_tests.cpp
+++ b/test/mp/test/listen_tests.cpp
@@ -168,6 +168,12 @@ public:
return connected_count;
}
+ size_t DisconnectedCount()
+ {
+ std::lock_guard<std::mutex> lock(counter_mutex);
+ return disconnected_count;
+ }
+
void WaitForConnectedCount(size_t expected_count)
{
std::unique_lock<std::mutex> lock(counter_mutex);
@@ -203,8 +209,8 @@ public:
KJ_TEST("ListenConnections accepts incoming connections")
{
ListenSetup server;
+ KJ_EXPECT(server.ConnectedCount() == 0);
auto client = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
-
server.WaitForConnectedCount(1);
KJ_EXPECT(client->client->add(1, 2) == 3);
}
@@ -217,29 +223,34 @@ KJ_TEST("ListenConnections enforces a local connection limit")
ListenSetup server(/*max_connections=*/1);
+ KJ_EXPECT(server.ConnectedCount() == 0);
auto client1 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
server.WaitForConnectedCount(1);
+
KJ_EXPECT(client1->client->add(1, 2) == 3);
auto client2 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
-
// Without this sync, ConnectedCount() == 2 might pass even if
// max_connections was not enforced because the event loop has not accepted
// client3 yet.
(**server.m_loop_ref).sync([] {});
- KJ_EXPECT(server.ConnectedCount() == 1);
+ KJ_EXPECT(server.ConnectedCount() == 1);
+ KJ_EXPECT(server.DisconnectedCount() == 0);
client1.reset();
server.WaitForDisconnectedCount(1);
server.WaitForConnectedCount(2);
KJ_EXPECT(client2->client->add(2, 3) == 5);
+ KJ_EXPECT(server.DisconnectedCount() == 1);
client2.reset();
server.WaitForDisconnectedCount(2);
+ KJ_EXPECT(server.ConnectedCount() == 2);
auto client3 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
server.WaitForConnectedCount(3);
+
KJ_EXPECT(client3->client->add(3, 4) == 7);
}
@@ -250,22 +261,22 @@ KJ_TEST("ListenConnections accepts multiple connections")
ListenSetup server(/*max_connections=*/2);
+ KJ_EXPECT(server.ConnectedCount() == 0);
auto client1 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
auto client2 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
-
server.WaitForConnectedCount(2);
KJ_EXPECT(client1->client->add(1, 2) == 3);
KJ_EXPECT(client2->client->add(2, 3) == 5);
auto client3 = std::make_unique<ClientSetup>(server.listener.MakeConnectedSocket());
-
// Without this sync, ConnectedCount() == 2 might pass even if
// max_connections was not enforced because the event loop has not accepted
// client3 yet.
(**server.m_loop_ref).sync([] {});
- KJ_EXPECT(server.ConnectedCount() == 2);
+ KJ_EXPECT(server.ConnectedCount() == 2);
+ KJ_EXPECT(server.DisconnectedCount() == 0);
client1.reset();
server.WaitForDisconnectedCount(1);
server.WaitForConnectedCount(3);There was a problem hiding this comment.
Done. added this helper as well as included in the relevant tests
Thanks for giving this a look, i intend to make another round of updates. Should be able to have that done soon |
This bumps the major version to 12 because upcoming commits introduce a non-trivial feature by adding a local max-connections parameter to the ListenConnections() method This also records release notes for v11 in doc/version.md. These notes are unrelated to this PR and describe changes that will be tagges before this PR
683bee9 to
e4dad41
Compare
|
Thanks for the reviews @ryanofsky @Eunovo. Addressed them in the latest commits
This would be helpful to bitcoin/bitcoin#35037 as well as greatly simplify the work needed for the PR. Looking forward to reviewing this |
95aaf9e to
f9ef92e
Compare
Add a separate listen_tests.cpp file with reusable UnixListener, ClientSetup and ListenSetup helpers for exercising ListenConnections() with real Unix domain sockets. The new test covers the baseline behavior that ListenConnections() accepts an incoming connection and serves requests over it. Keeping this coverage separate from the existing general proxy tests makes the socket listener setup easier to review and provides a clearer place to extend listener-specific behavior in follow-up commits.
f9ef92e to
e63835c
Compare
Add an optional max_connections parameter to ListenConnections() and track the limit with listener-local active connection state, so accepting pauses at capacity and resumes after a disconnect. Update listener tests for cap enforcement, resume behavior, and multiple active connections.
e63835c to
39a10ce
Compare
Addressed these in the recent change. Faced some IWYU errors after the comment above. So had to resolve them anyway. Thanks |
|
Post-merge re-ACK 39a10ce |
This adds an optional local connection limit to
ListenConnections().Previously,
ListenConnections()would accept incoming connections indefinitely. This branch adds an optionalmax_connectionsparameter so a listener can stop accepting new connections once a per-listener cap is reached, and resume accepting when an existing connection disconnects.The limit is local to the listener instead of global to the
EventLoop. This keeps the state and behavior scoped to the listening socket, and is closer to the direction discussed downstream for per--ipcbindlimits.This also adds a test covering the behavior with
max_connections=1, verifying that:Note This PR includes a major version bump to
v12due to the API addition. If #274 lands earlier and bumps the version tov12first, we will need to bump the version here again.