Shut down gracefully, and clean up when starting up fails

CHANGES.txt

@@ -1,9 +1,64 @@
 Unreleased
 ----------
 
+Features
+~~~~~~~~
+
+- Waitress now shuts down gracefully. Upon being interrupted it closes its
+  listening sockets, stops reading new requests off the connections that are
+  still open, and keeps running the main loop until the requests that are
+  already being serviced have finished and their responses have been written
+  back to the client. Only then are the worker threads stopped and the
+  remaining connections closed.
+
+  Previously the main loop was stopped immediately, which meant that any
+  response that had not been fully written out yet was dropped, and that a
+  task thread that was waiting on the main loop to drain an output buffer was
+  left blocked forever.
+
+  This is also exposed as ``server.graceful_shutdown()`` for those that drive
+  the server themselves instead of using ``server.run()``. See
+  https://github.com/Pylons/waitress/issues/198 and
+  https://github.com/Pylons/waitress/issues/264
+
+- Servers now have public ``start()`` and ``stop()`` methods that run the main
+  loop in a background thread and shut it down again. ``stop()`` is safe to
+  call from any thread, including from within the WSGI application. This
+  removes the need for the kind of workaround that WebTest's
+  ``StopableWSGIServer`` had to resort to, and combined with
+  ``server.effective_port`` it allows binding to port ``0`` and reading back
+  the port that was picked without racing anybody else for it. See
+  https://github.com/Pylons/waitress/issues/290
+
+- Added a new ``shutdown_timeout`` adjustment (default: 5 seconds) that bounds
+  how long the graceful shutdown described above may take, and that replaces
+  the previously hardcoded timeout used when stopping the worker threads. Set
+  it to ``0`` to tear everything down immediately instead. See
+  https://github.com/Pylons/waitress/issues/134
+
 Bugfix
 ~~~~~~
 
+- Waitress no longer leaks threads, sockets and socket map entries when it
+  fails to create a server, for example because the port it was asked to
+  listen on is already in use. The worker threads are no longer started until
+  every listening socket has been successfully bound, and anything that was
+  created before the failure is now cleaned up rather than being left running
+  with no way for the caller to get at it. See
+  https://github.com/Pylons/waitress/issues/480
+
+- ``BaseWSGIServer.close()`` now shuts down the task dispatcher and closes the
+  connections that are still open, matching what ``MultiSocketServer.close()``
+  already did. Previously closing a server that was listening on a single
+  socket left its worker threads running. See
+  https://github.com/Pylons/waitress/issues/402 and
+  https://github.com/Pylons/waitress/issues/480
+
+- Pulling the trigger after it has been closed is now a no-op instead of an
+  error. A task thread that finished right as the server was going away could
+  previously write to a file descriptor that had already been closed, and
+  potentially handed out to something else in the meantime.
+
 - Renamed the HTTP header "Trailers" to "Trailer" to fix a typo and comply with
   the correct header name as specified in RFC 7230.
 

docs/arguments.rst

@@ -270,6 +270,19 @@ channel_timeout
 
     Default: ``120``
 
+shutdown_timeout
+    Maximum seconds to spend shutting down gracefully (integer).  Once waitress
+    is asked to stop it stops accepting new connections and keeps running its
+    main loop for at most this long, so that requests that are already being
+    serviced get a chance to finish and have their response written back to the
+    client.  Connections that are still open once this expires are closed, and
+    any tasks that are still queued are cancelled.  Set to ``0`` to tear
+    everything down immediately instead.
+
+    Default: ``5``
+
+    .. versionadded:: 3.1.0
+
 log_socket_errors
     Set to ``False`` to not log premature client disconnect tracebacks.
 

docs/runner.rst

@@ -264,6 +264,14 @@ Tuning options:
     120. 'Inactive' is defined as 'has received no data from the client and has
     sent no data to the client'.
 
+``--shutdown-timeout=INT``
+    Maximum number of seconds to spend shutting down gracefully.  Default is 5.
+    Once waitress is asked to stop it stops accepting new connections and keeps
+    running its main loop for at most this long, so that requests that are
+    already being serviced get a chance to finish and have their response
+    written back to the client.  Set to ``0`` to tear everything down
+    immediately instead.
+
 ``--channel-request-lookahead=INT``
     Sets the amount of requests we can continue to read from the socket, while
     we are processing current requests. The default value won't allow any

docs/usage.rst

@@ -80,6 +80,88 @@ can be used in development and in situations where the likes of
     # Listen on only IPv4 on port 8041
     waitress-serve --port=8041 myapp:wsgifunc
 
+.. _shutdown:
+
+Shutting down
+-------------
+
+When ``server.run()`` is interrupted, either by :kbd:`Ctrl-C` or by a
+``SystemExit``, Waitress shuts down gracefully:
+
+- it closes its listening sockets, so the port is released and no new
+  connections are accepted;
+- it stops reading new requests off the connections that are still open;
+- it keeps running its main loop so that requests that are already being
+  serviced can finish and have their response written back to the client;
+- once every connection has been dealt with, it stops the worker threads and
+  closes up shop.
+
+Requests that have not finished within ``shutdown_timeout`` seconds (5 by
+default) do not get to hold up the shutdown any further, and their connections
+are closed. Interrupting a second time skips the wait entirely. See
+:ref:`arguments` for ``shutdown_timeout``.
+
+Waitress does not install any signal handlers of its own. To shut down on a
+``SIGTERM`` from a process manager, install a handler that raises
+``SystemExit`` in the main thread and let ``run()`` take it from there:
+
+.. code-block:: python
+
+    import signal
+    import sys
+
+    from waitress.server import create_server
+
+    server = create_server(app, listen="*:8080")
+    signal.signal(signal.SIGTERM, lambda *args: sys.exit(0))
+    server.run()
+
+The shutdown is also available directly as ``server.graceful_shutdown()``, for
+when you drive the main loop yourself rather than through ``run()``. Note that
+it runs the main loop while it drains, so it has to be called from the same
+thread the loop runs on.
+
+``server.close()`` is the non-graceful counterpart: it stops the worker threads
+and closes every connection immediately, without giving the requests that are
+in flight a chance to finish.
+
+.. _running-in-a-thread:
+
+Running in a background thread
+------------------------------
+
+``server.run()`` blocks. To run a server alongside something else, for example
+to serve an application under test, use ``start()`` and ``stop()`` instead:
+
+.. code-block:: python
+
+    from waitress.server import create_server
+
+    server = create_server(app, host="127.0.0.1", port=0)
+    server.start()
+    try:
+        ...
+    finally:
+        server.stop()
+
+``start()`` runs the main loop in a daemon thread and returns immediately.
+``stop()`` shuts the server down gracefully and waits for it to finish, and is
+the one method that may be called from any thread, including from within the
+WSGI application itself.
+
+Because the server is already listening by the time ``start()`` returns, asking
+for port ``0`` and then reading back the port that the operating system picked
+is race free — there is no window in which another process could take it:
+
+.. code-block:: python
+
+    server = create_server(app, host="127.0.0.1", port=0)
+    server.start()
+    url = f"http://{server.effective_host}:{server.effective_port}/"
+
+With several listening sockets, ``server.effective_listen`` holds the full list
+of ``(host, port)`` pairs instead.
+
 Heroku
 ------
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "waitress"
-version = "3.0.2"
+version = "3.1.0.dev0"
 authors = [{ name = "Zope Foundation and Contributors", email = "zope-dev@zope.org" }]
 maintainers = [{ name = "Pylons Project", email = "pylons-discuss@googlegroups.com" }]
 description = "Waitress WSGI server"

src/waitress/adjustments.py

@@ -138,6 +138,7 @@ class Adjustments:
         ("connection_limit", int),
         ("cleanup_interval", int),
         ("channel_timeout", int),
+        ("shutdown_timeout", int),
         ("log_socket_errors", asbool),
         ("max_request_header_size", int),
         ("max_request_body_size", int),
@@ -253,6 +254,14 @@ class Adjustments:
     # Maximum seconds to leave an inactive connection open.
     channel_timeout = 120
 
+    # Maximum seconds to spend shutting down gracefully. Once the server is
+    # asked to stop it no longer accepts new connections, and keeps running the
+    # main loop for at most this long so that requests that are already being
+    # serviced get a chance to finish and have their response written back to
+    # the client. Set to 0 to disable the graceful shutdown entirely and tear
+    # everything down immediately.
+    shutdown_timeout = 5
+
     # Boolean: turn off to not log premature client disconnects.
     log_socket_errors = True
 

src/waitress/channel.py

@@ -48,6 +48,7 @@ class HTTPChannel(wasyncore.dispatcher):
     sent_continue = False  # used as a latch after sending 100 continue
     total_outbufs_len = 0  # total bytes ready to send
     current_outbuf_count = 0  # total bytes written to current outbuf
+    draining = False  # set to True to stop reading new requests off the socket
 
     #
     # ASYNCHRONOUS METHODS (including __init__)
@@ -142,13 +143,17 @@ def readable(self):
         # 1. We're not already about to close the connection.
         # 2. We're not waiting to flush remaining data before closing the
         #    connection
-        # 3. There are not too many tasks already queued (if lookahead is enabled)
-        # 4. There's no data in the output buffer that needs to be sent
+        # 3. We're not draining the channel for a graceful shutdown, during
+        #    which we finish the requests we already have but don't take on
+        #    any new ones.
+        # 4. There are not too many tasks already queued (if lookahead is enabled)
+        # 5. There's no data in the output buffer that needs to be sent
         #    before we potentially create a new task.
 
         return not (
             self.will_close
             or self.close_when_flushed
+            or self.draining
             or len(self.requests) > self.adj.channel_request_lookahead
             or self.total_outbufs_len
         )

src/waitress/runner.py

@@ -232,6 +232,14 @@
         Default is 120. 'Inactive' is defined as 'has received no data
         from the client and has sent no data to the client'.
 
+    --shutdown-timeout=INT
+        Maximum number of seconds to spend shutting down gracefully.
+        Default is 5. Once waitress is asked to stop it stops accepting
+        new connections and keeps running its main loop for at most this
+        long, so that requests that are already being serviced get a
+        chance to finish and have their response written back to the
+        client. Set to '0' to tear everything down immediately instead.
+
     --channel-request-lookahead=INT
         Sets the amount of requests we can continue to read from the socket,
         while we are processing current requests. The default value won't allow