From 37a95b49d11e5213660dd17cc3b2bf168613e2fd Mon Sep 17 00:00:00 2001 From: qiulang2000 Date: Sat, 3 Oct 2026 11:29:35 +0800 Subject: [PATCH 1/3] doc: using docker swarm Why Swarm is a better choice than Kubernetes for a cluster of a dozen or so machines --- .../getting_started/using_docker_swarm.rst | 497 ++++++++++++++++++ 1 file changed, 497 insertions(+) create mode 100644 doc/source/getting_started/using_docker_swarm.rst diff --git a/doc/source/getting_started/using_docker_swarm.rst b/doc/source/getting_started/using_docker_swarm.rst new file mode 100644 index 0000000000..cecbeac20a --- /dev/null +++ b/doc/source/getting_started/using_docker_swarm.rst @@ -0,0 +1,497 @@ +.. _using_docker_swarm: + +======================================== +Distributed Deployment with Docker Swarm +======================================== + +This document covers three topics: + +1. Why Docker Swarm is a better choice than Kubernetes for a cluster of a dozen or so machines +2. Understanding Swarm by comparing it with Xinference's supervisor/worker +3. How to adapt the official compose file for Swarm, with an introduction to basic Swarm concepts and commands + +Tested with Xinference 2.2.0, Docker 29.2.1, Ubuntu 22.04, and NVIDIA driver 570 on 3 machines, each with one NVIDIA L20. The adjustments for Xinference 3.x in Section 4.8 are based on the official documentation and source code and have not yet been verified in a deployment. + +.. note:: + + Both Swarm and Xinference have a role called "worker", but the term refers to different things. In this document, "Swarm node" means a physical machine that has joined the Swarm cluster, and "Xinference worker" means the Xinference process that runs inference. + +1. Why Swarm is a better choice than Kubernetes for a cluster of a dozen or so machines +================================================= + +Our setup is a dozen or so GPU servers, maintained by a small team, mainly running Xinference. What we need from an orchestration tool: + +- Deploy containers across multiple machines +- Restart containers automatically when they exit abnormally +- Let containers on different machines reach each other +- Assign Xinference workers according to the number of GPUs +- Upgrade versions across all machines in one step + +Swarm handles all of these well, although GPU allocation requires extra manual configuration (see Section 1.1). + +Advantages of Swarm: + +- It is built into Docker, so there is nothing extra to install. ``docker swarm init`` creates a cluster and ``docker swarm join`` adds a machine +- The deployment file is a docker-compose file plus a ``deploy`` section, so anyone who can write compose files has little new to learn +- Deployment is a single command: ``docker stack deploy -c docker-compose-swarm.yml xinference`` + +What Kubernetes adds on top, such as load-based autoscaling, multi-tenant access control, and the Helm and Operator ecosystem, is not needed at this scale. In exchange, it brings learning and maintenance costs: etcd, the API server, the scheduler, network plugins, and GPU device plugins each need someone who understands them, and each of them can fail. For a small team, Kubernetes is overkill. + +Swarm does have limitations to be aware of: + +- **GPU support is Swarm's main weakness.** All of our machines have GPUs, so this directly affects deployment and operations. It is covered separately in Section 1.1 +- We use a single manager. While the manager machine is down, services cannot be deployed, upgraded, or adjusted; containers that are already running are not affected +- There are fewer community resources than for Kubernetes, so less help is available when problems come up + +If you later need to share GPUs among multiple teams or autoscale based on load, or if your organization already runs a Kubernetes cluster, Xinference also documents a Kubernetes deployment (see :ref:`using_kubernetes`), and you can migrate then. + +1.1 GPU Support: What to Know Before Choosing Swarm +--------------------------------------------------- + +**Swarm does not know about GPUs** + +Kubernetes itself does not know about GPUs either, but the agent Kubernetes runs on each node (kubelet) defines a gRPC interface called the device plugin interface. Any vendor can write a program that implements this interface to report which devices a node has and whether each one is healthy, and to decide which device a container gets when it starts. NVIDIA implements this interface in the NVIDIA device plugin. Once deployed on each node, the plugin detects the node's GPUs automatically, reports them, and marks a GPU as unavailable when it fails. The plugin itself has to be installed, and each node still needs the driver and the NVIDIA Container Toolkit. + +Swarm has no such interface. It only has a general-purpose counting mechanism called generic resources: + +1. When Docker starts, it reads ``node-generic-resources`` from ``/etc/docker/daemon.json``, written there by hand, and reports the values to the manager. The values do not change after that. Swarm does not know what these values represent; it treats them as a set of strings that can be allocated +2. At deployment time, Swarm picks one of the values for a container and passes it in through the environment variable ``DOCKER_RESOURCE_NVIDIA-GPU=`` +3. For the container to actually see only that GPU, nvidia-container-runtime has to read this environment variable. This requires setting ``swarm-resource = "DOCKER_RESOURCE_GPU"`` in ``/etc/nvidia-container-runtime/config.toml`` + +Swarm and NVIDIA work together through an agreed-upon environment variable name, not through a formal plugin mechanism. + +**Comparison** + +.. list-table:: + :header-rows: 1 + + * - + - Kubernetes + NVIDIA device plugin + - Swarm + * - GPU discovery + - Detected automatically by the plugin + - Written by hand into ``daemon.json`` on each machine + * - Configuration needed + - Install the plugin (or have the NVIDIA GPU Operator install it) + - Edit ``daemon.json`` and ``config.toml`` on each machine, then restart Docker + * - GPU failure + - The plugin marks the GPU unhealthy and stops assigning it + - Swarm is unaware and keeps assigning containers to it + * - Replacing a GPU + - Updated automatically by the plugin + - The UUID changes; reconfigure and restart Docker + * - Identification + - Handled by the plugin + - GPU UUIDs must be entered by hand; GPU indices do not work + +**What this means in practice** + +- Every machine needs two extra configuration steps before deployment. If they are skipped or done wrong, the Xinference worker stays Pending with ``insufficient resources``. We ran into this during deployment: declaring the GPU by index (``NVIDIA-GPU=0``) caused this error, removing ``node-generic-resources`` caused the same error, and it only worked after switching to GPU UUIDs +- These two steps must not be skipped when adding a machine; otherwise the worker does not start after ``docker swarm join`` +- After a GPU is replaced, the configuration must be redone +- Swarm does not notice when a GPU fails. You have to detect the problem yourself through ``nvidia-smi`` or the Xinference Web UI, then take that machine out of the cluster (see "GPU failure" in the deployment guide) +- The number of GPUs per machine determines how Xinference workers are laid out. With one GPU per machine, ``mode: global`` is the simplest; machines with multiple GPUs need a different approach (see "Multi-GPU machines" in the deployment guide) + +We wrote a script, ``setup-gpu-swarm.sh``, that gets the GPU UUIDs and edits both configuration files automatically. It reduces manual work, but it still has to be run once on every machine, and again after a GPU is replaced. + +**Why we still chose Swarm** + +Our cluster has a dozen or so machines with identical configurations, and GPUs are rarely replaced. The manual configuration is a one-time task, and with the script it takes a few minutes per machine. Compared with deploying and maintaining a Kubernetes cluster, this cost is acceptable. + +If the GPU models or counts in your cluster change often, or you need GPU failures to be handled automatically, Kubernetes has a clear advantage here. + +2. Understanding Swarm Through Xinference's Supervisor/Worker +============================================================= + +2.1 Similar Structure, Different Responsibilities +------------------------------------------------- + +Both Swarm and Xinference follow a "one control node + multiple execution nodes" structure. If you are familiar with Xinference, you can use its supervisor/worker to understand Swarm's manager/node: + +.. list-table:: + :header-rows: 1 + + * - + - Swarm + - Xinference + * - Control role + - manager + - supervisor + * - Execution role + - Swarm node (physical machine) + - Xinference worker (process in a container) + * - What it schedules + - Containers + - Models + * - Resources it tracks + - CPU, memory, and GPUs of each machine + - GPU memory and model usage + +Swarm decides which machine a container runs on, without knowing what runs inside the container. Xinference decides which worker loads a model and where inference requests go, but it does not start worker processes. In our deployment, these two layers work on top of each other. + +The two layers meet at the GPU: Swarm assigns a GPU to a Xinference worker container, and the container can see only that GPU; the Xinference worker detects that GPU at startup and loads models on it. + +2.2 The Key Difference: The Supervisor Is Passive, the Manager Is Active +------------------------------------------------------------------------ + +The similar structure raises a common question: if Xinference already has supervisor/worker, why use Swarm? + +The answer lies in how the two work. + +**Xinference's supervisor is passive.** It only accepts registrations: the worker process on each machine connects to the supervisor after starting, and only then does the supervisor know about that worker. The supervisor cannot start a worker on another machine. When a worker process exits, the supervisor can only remove it from its list; it cannot bring it back up. + +**Swarm's manager is active.** It makes sure these processes exist: which machine to start each container on, with which image and arguments, and restarting containers when they fail. + +So without Swarm, Xinference's supervisor/worker still works, but someone has to do the following by hand: + +- Log in to each machine and run ``docker run`` with the correct supervisor address, local IP, port, and other arguments +- When upgrading Xinference, stop the container, pull the image, and restart it on each machine in turn +- When adding a machine, deploy a worker on it manually +- Log in to each machine to check status and logs +- When the machine running the supervisor goes down, restart the supervisor on another machine manually + +With Swarm: + +- Run ``docker stack deploy`` once on the manager, and the supervisor and workers on all machines are deployed together +- When a container exits abnormally, Swarm restarts it automatically +- After a new machine runs ``docker swarm join``, a Xinference worker is deployed on it automatically +- To upgrade, change the image version in the compose file and run ``docker stack deploy`` again +- Use ``docker service ps`` and ``docker service logs`` on the manager to check status and logs across all machines + +In short: Xinference manages models, and Swarm manages containers. + +2.3 What Swarm Does Not Solve +----------------------------- + +- When the machine running the supervisor goes down, Swarm restarts the supervisor on another machine, and Xinference workers can still find it through the service name. However, the new supervisor does not keep records of previously loaded models, and we have not verified whether Xinference workers re-register automatically +- If the machine that goes down happens to be the manager, Swarm itself cannot reschedule anything + +With only two or three machines, manual maintenance is not costly. The more machines there are, the more manual work Swarm saves. + +3. Swarm Basics +=============== + +3.1 Node Roles +-------------- + +A Swarm cluster has two kinds of machines: + +- **manager**: the machine that runs ``docker swarm init``. It handles scheduling and management. All commands for deploying and inspecting services are run on the manager +- **Regular node**: a machine that joins with ``docker swarm join``. It runs containers + +The manager also runs containers, just like regular nodes. + +Swarm supports multiple managers that back each other up (3 managers can tolerate the failure of 1); we use only one for simplicity. The token used to join decides the role: ``docker swarm join-token worker`` generates the token for a regular node, and ``docker swarm join-token manager`` generates the token for a manager. + +3.2 Key Concepts +---------------- + +.. list-table:: + :header-rows: 1 + + * - Concept + - Description + * - Stack + - A group of services deployed from one compose file with ``docker stack deploy``. Our stack is named ``xinference`` + * - Service + - Each service in the compose file, such as ``xinference-supervisor``. After deployment, the stack name is added as a prefix: ``xinference_xinference-supervisor`` + * - Task + - A running instance of a service. Normally each replica is one task. When a container fails and restarts, the old task is marked Failed and Swarm creates a new task to replace it. Each line in the output of ``docker service ps`` is a task + * - Container + - The container that a task actually runs on a machine + * - Overlay network + - A virtual network that spans machines; containers on different machines reach each other by service name + * - Ingress + - An overlay network that Swarm creates automatically to forward published ports to containers + +Hierarchy: Stack → Service → Task → Container + +The same replica slot may have several task records (the current one and earlier failed ones), but at any moment only one task is running, corresponding to one running container. + +You do not need to create or manage tasks, but you should be able to read the output of ``docker service ps`` when troubleshooting: + +.. code-block:: text + + ID NAME NODE DESIRED STATE CURRENT STATE ERROR + rdr5avu6kz8o xinference_xinference-worker.xx node-1 Running Running + v3lu6wcwdtuj \_ xinference_xinference-worker.xx node-1 Shutdown Failed "task: non-zero exit (1)" + +Lines with ``\_`` are earlier tasks in the same slot, and the ``NODE`` column shows the machine the container is on. + +3.3 Basic Commands +------------------ + +**Cluster management** + +.. code-block:: bash + + # On the manager: create the cluster + docker swarm init --advertise-addr + + # On the manager: show the command for joining the cluster + docker swarm join-token worker + + # On each other machine: join the cluster (copy the command from the previous step) + docker swarm join --token SWMTKN-1-xxxxx :2377 + + # On the manager: list all nodes + docker node ls + + # On a machine: leave the cluster + docker swarm leave + + # On the manager: remove the record of a node that has left + docker node rm + +``--advertise-addr`` tells other machines which IP to use to reach the manager. It can be omitted if the machine has only one network interface; with multiple network interfaces it is required, otherwise an error is reported. + +**Deploying and inspecting services** (run on the manager) + +.. code-block:: bash + + # Deploy or update (run again after editing the compose file) + docker stack deploy -c docker-compose-swarm.yml xinference + + # List all services and their replica counts + docker service ls + + # Show the tasks of a service: which machine and what state + docker service ps xinference_xinference-worker + + # Show full error messages (truncated by default) + docker service ps --no-trunc xinference_xinference-worker + + # Show logs, aggregated from the containers on all machines + docker service logs xinference_xinference-worker + docker service logs -f xinference_xinference-worker # follow + + # Remove the whole stack + docker stack rm xinference + +**Entering a container** + +.. code-block:: bash + + # First, on the manager, use docker service ps to find the machine (NODE column) + # Then log in to that machine and run + docker ps + docker exec -it bash + +``docker stack``, ``docker service``, and ``docker node`` commands can only be run on the manager; ``docker ps`` and ``docker exec`` must be run on the machine where the container is running. + +3.4 Compose Files Under Swarm +----------------------------- + +The same compose file behaves differently when deployed with ``docker compose up`` and with ``docker stack deploy``: + +- ``depends_on`` is ignored. Swarm does not guarantee the start order of services, so applications have to retry on their own +- ``restart: always`` has no effect; use ``deploy.restart_policy`` instead +- ``build`` has no effect. Only prebuilt images can be used, and every machine must be able to pull them +- The ``deploy`` section only takes full effect under Swarm: replica count (``replicas``), one per machine (``mode: global``), resource reservations, placement constraints, and so on are all set here +- Configuration files are distributed with ``configs``. Swarm sends the file contents to the machine where the container runs, so you do not need a copy on every machine + +4. Adapting the Official Compose File +===================================== + +4.1 Starting Point +------------------ + +The starting point is `docker-compose-distributed.yml `__ in the Xinference repository. + +This file defines one supervisor and two workers. ``docker compose`` runs on a single machine only, so the file actually demonstrates the distributed structure on one machine. The comments in the file also say it is an example with two workers, and more can be added by incrementing the numbering. + +The repository also has a ``docker-compose.yml``. It is for standalone deployment: there is only one container, and the supervisor and worker run in the same process (``xinference-local``). It is not a suitable basis for a distributed deployment. + +When Xinference 3.0 was released, the standalone ``docker-compose.yml`` was updated, but the distributed file was not (its health check still uses curl and its image is still ``latest``). So the adjustments needed for 3.x are based on the standalone file and the 3.0 migration guide. + +4.2 The Supervisor Does Not Need a GPU +-------------------------------------- + +The original file uses a YAML anchor that makes the supervisor inherit the GPU configuration: + +.. code-block:: yaml + + xinference: &xinference + deploy: + resources: + reservations: + devices: + - capabilities: [gpu] + + xinference-supervisor: + <<: *xinference # inherits the GPU configuration + +The supervisor only schedules; it does not run inference. The source code confirms this: the supervisor's startup code creates only a scheduling actor and an HTTP service, with no GPU-related code, while the worker's startup code detects the number of GPUs and passes the GPU list to the worker. This is a side effect of reusing the template. After the change, the supervisor does not request a GPU. + +4.3 Merging the Workers into One Service +---------------------------------------- + +The original file defines each worker as a separate service (``xinference-worker-1``, ``xinference-worker-2``), each with its own port. After the change, there is a single ``xinference-worker`` service with ``mode: global``, so Swarm runs one on every machine. When a new machine joins the cluster, a worker is deployed on it automatically, without editing the compose file. + +``mode: global`` suits machines with one GPU each. If machines have multiple GPUs, the worker layout needs to be reconsidered; see "Multi-GPU machines" in the deployment guide. + +4.4 Changing ``--host`` to ``$(hostname -i)`` +--------------------------------------------- + +This is the most important change. First, look at how Xinference registration works. + +When a worker starts, it sends its own address (``--host`` plus ``--worker-port``) to the supervisor: + +.. code-block:: python + + # worker side + await self._supervisor_ref.add_worker(self.address) + +When the supervisor receives it, it uses this address to connect back to the worker: + +.. code-block:: python + + # supervisor side + async def add_worker(self, worker_address: str): + worker_ref = await xo.actor_ref(address=worker_address, uid=WorkerActor.default_uid()) + +The same applies in the other direction: at startup, the worker first asks the supervisor for its internal address through the REST API, then connects to the supervisor using that address. + +So the value of ``--host`` **must be an address the other side can connect back to**. + +**Why the original file can use hostnames on a single machine** + +.. code-block:: bash + + xinference-worker -H xinference-worker-1 --worker-port 30001 + xinference-worker -H xinference-worker-2 --worker-port 30002 + +In a single-machine compose deployment, each service is defined separately and the service name is the hostname, which Docker DNS resolves to a unique container IP. All containers are on the same network and can reach each other. + +**Neither condition holds under Swarm** + +Once the workers become one service, all replicas share the service name ``xinference-worker`` and have no hostnames of their own. Swarm DNS offers only two kinds of resolution: ``xinference-worker`` resolves to a virtual IP that Swarm load-balances to one of the replicas, and ``tasks.xinference-worker`` returns the IPs of all replicas. Neither can point to a specific replica. + +If ``-H 0.0.0.0`` is used instead, every worker registers the address ``0.0.0.0:30001``, and when the supervisor connects back to that address, it connects to itself. + +The supervisor has the same problem. With ``--host 0.0.0.0``, the internal address the supervisor gives to workers is ``0.0.0.0:9999``, and a worker connecting to it connects to itself. This is the log from an actual deployment: + +.. code-block:: text + + xinference.core.worker ERROR Failed to upload node info: [Errno 111] Connect call failed ('0.0.0.0', 9999) + +At that point, the cluster information page of the Web UI showed 0 workers. + +**Solution** + +Inside the container, use ``hostname -i`` to get the container's IP on the overlay network, and start with that IP: + +.. code-block:: yaml + + command: sh -c 'xinference-supervisor --host $$(hostname -i) --port 9997 --supervisor-port 9999' + command: sh -c 'xinference-worker -e http://xinference-supervisor:9997 --host $$(hostname -i) --worker-port 30001' + +Each container has a unique IP on the overlay network that the others can reach, with no dependence on DNS. ``$$`` is an escape in compose files; the shell receives ``$(hostname -i)``. + +After the change, the supervisor address is an overlay IP such as ``10.0.1.3:9999``, and the 3 workers register as ``10.0.1.6:30001``, ``10.0.1.7:30001``, and ``10.0.1.8:30001``. + +4.5 Why nginx Is Needed +----------------------- + +After ``--host`` is changed to the overlay IP, workers can register, but the Web UI can no longer be reached from outside. The reason is that ``--host`` controls two things at once: + +- **The REST API listening address** (port 9997): the Web UI and HTTP API, which must be reachable from outside +- **The actor address** (port 9999): internal communication between the supervisor and workers, which must give workers an address they can connect back to + +.. code-block:: python + + # supervisor startup code + supervisor_address = f"{host}:{supervisor_port}" # actor address + restful_api.run(supervisor_address=supervisor_address, host=host, port=port) # the REST API uses the same host + +The two have conflicting requirements for the address: + +- ``--host 0.0.0.0``: the REST API listens on all network interfaces and is reachable from outside, but the actor address becomes ``0.0.0.0:9999`` and worker registration fails +- ``--host ``: worker registration works, but the REST API listens only on the overlay network + +Why does listening only on the overlay network make it unreachable from outside? In Swarm, a container that publishes a port has two network interfaces: one on the ingress network and one on the overlay network we defined (``xinference-net``). External traffic to ``:9997`` enters the container through the ingress network, but the REST API listens only on the overlay IP, so it never receives that traffic. + +A single-machine compose deployment does not have this problem: the container has only one network interface, so the port forwarding target and the address the service listens on are the same IP. + +Xinference's command line has no separate option for the REST API listening address (as of 3.5.0, the supervisor's only options are ``--host``, ``--port``, ``--supervisor-port``, and ``--log-level``), so an nginx container is added to forward traffic: + +- The supervisor does not publish a port and starts with its overlay IP, so internal communication works +- nginx publishes port 9997, listens on all network interfaces, and receives external traffic +- nginx forwards requests over the overlay network to the supervisor, using the service name ``xinference-supervisor`` + +.. code-block:: text + + Browser → any machine IP:9997 → ingress network → nginx container → overlay network → supervisor (10.0.1.x:9997) + +Ports published by Swarm are reachable on every machine in the cluster, so the Web UI can be opened through port 9997 on any machine's IP. + +If Xinference later provides separate options for the REST API listening address and the actor address, nginx can be removed. + +4.6 Requesting GPUs +------------------- + +The background for this change is in Section 1.1: Swarm does not know about GPUs, so GPUs can only be declared and allocated by hand through generic resources. + +On a single Docker host, GPUs can be used directly once the NVIDIA Container Toolkit is installed, and the original file uses this single-host syntax: + +.. code-block:: yaml + + # Original file: single-host docker compose syntax + deploy: + resources: + reservations: + devices: + - capabilities: [gpu] + driver: nvidia + count: all + +A single host involves no scheduling; Docker mounts the local GPUs into the container directly. Swarm has to decide which machine each container goes on, so it needs to know how many GPUs each machine has and how many are already assigned. That is why the syntax changes to generic resources: + +.. code-block:: yaml + + # After the change: Swarm syntax + deploy: + resources: + reservations: + generic_resources: + - discrete_resource_spec: + kind: 'NVIDIA-GPU' + value: 1 # each worker requests 1 GPU + +``kind`` must match the name used in ``node-generic-resources`` in each machine's ``daemon.json`` (``NVIDIA-GPU``). When scheduling, Swarm matches the number of GPUs declared on each machine against the number each worker requests; machines with no GPUs left are not assigned a worker. + +The compose file is only half of this mechanism. The other half is in ``daemon.json`` and ``config.toml`` on each machine, and both halves must be configured for it to work. + +4.7 Start Order +--------------- + +Swarm ignores ``depends_on``, so the supervisor and workers start at almost the same time. If a worker starts before the supervisor is ready, the connection fails and the container exits. + +This is handled by the restart policy: the worker's ``restart_policy.delay`` is set to 20 seconds, so after a failure it waits 20 seconds before restarting, by which time the supervisor is up. In our tests, with 10 seconds each worker failed four times before succeeding; with 20 seconds it succeeded on the first try. + +4.8 Adjustments for Xinference 3.x +---------------------------------- + +Several changes in Xinference 3.0 affect deployment. The following adjustments were made on 3.5.0: + +.. list-table:: + :header-rows: 1 + + * - Change + - Adjustment + * - The official GPU image is based on CUDA 13 and requires NVIDIA driver 580 or later + - Upgrade the driver on all machines + * - Authentication is enabled by default; every API call requires a login or an API key + - For use on an internal network, disable it with ``XINFERENCE_AUTH_ADVANCED=false`` + * - The image no longer bundles inference engines such as vLLM and SGLang; they are installed into ``XINFERENCE_HOME/virtualenv`` the first time a model is launched + - Mount the worker's ``/root/.xinference`` to a host directory so engines are not reinstalled when the container is recreated; configure a pip mirror reachable from your network + * - The official docs state that the image is guaranteed to contain python3, but not curl + - Use python3 for the health check + * - The Web UI moved from ``/ui`` to ``/`` + - Open the Web UI at ``http://:9997/`` + * - The official recommendation is to pin the image version in production + - Use ``xprobe/xinference:v3.5.0`` instead of ``latest``, so machines do not pull different versions at different times + +With authentication disabled, the only data the supervisor needs to persist is ``XINFERENCE_HOME/system-settings.json``, which holds model download settings (download source, pip index, and so on). If these settings are changed in the Web UI, they are lost when the supervisor is rescheduled to another machine. So keep these settings in environment variables in the compose file instead of changing them in the Web UI. That way, the supervisor needs no mounted directory and does not have to be pinned to a particular machine. + +If you later enable authentication, the authentication database and encryption key are stored under the supervisor's ``XINFERENCE_HOME/auth``. In that case, pin the supervisor to the manager (``placement.constraints: [node.role == manager]``, a built-in Swarm attribute that requires no labels) and mount ``/root/.xinference`` to a host directory. From cc7af6826daca79eff9093d38d725c74a56e2e7a Mon Sep 17 00:00:00 2001 From: qinxuye Date: Sun, 4 Oct 2026 14:25:44 +0800 Subject: [PATCH 2/3] docs: scope Swarm guide to deployment considerations --- doc/source/getting_started/index.rst | 1 + .../getting_started/using_docker_swarm.rst | 498 +----------------- .../getting_started/using_docker_swarm.po | 185 +++++++ .../getting_started/using_docker_swarm.po | 186 +++++++ .../getting_started/using_docker_swarm.po | 187 +++++++ .../getting_started/using_docker_swarm.po | 187 +++++++ .../getting_started/using_docker_swarm.po | 172 ++++++ .../getting_started/using_docker_swarm.po | 172 ++++++ .../getting_started/using_docker_swarm.po | 185 +++++++ .../getting_started/using_docker_swarm.po | 160 ++++++ .../getting_started/using_docker_swarm.po | 160 ++++++ 11 files changed, 1616 insertions(+), 477 deletions(-) create mode 100644 doc/source/locale/de/LC_MESSAGES/getting_started/using_docker_swarm.po create mode 100644 doc/source/locale/es/LC_MESSAGES/getting_started/using_docker_swarm.po create mode 100644 doc/source/locale/fr/LC_MESSAGES/getting_started/using_docker_swarm.po create mode 100644 doc/source/locale/it/LC_MESSAGES/getting_started/using_docker_swarm.po create mode 100644 doc/source/locale/ja/LC_MESSAGES/getting_started/using_docker_swarm.po create mode 100644 doc/source/locale/ko/LC_MESSAGES/getting_started/using_docker_swarm.po create mode 100644 doc/source/locale/pt_BR/LC_MESSAGES/getting_started/using_docker_swarm.po create mode 100644 doc/source/locale/zh_CN/LC_MESSAGES/getting_started/using_docker_swarm.po create mode 100644 doc/source/locale/zh_TW/LC_MESSAGES/getting_started/using_docker_swarm.po diff --git a/doc/source/getting_started/index.rst b/doc/source/getting_started/index.rst index 24d3bf8823..53abccccfb 100644 --- a/doc/source/getting_started/index.rst +++ b/doc/source/getting_started/index.rst @@ -13,6 +13,7 @@ Getting Started logging using_docker_image using_docker_compose + using_docker_swarm using_kubernetes troubleshooting environments diff --git a/doc/source/getting_started/using_docker_swarm.rst b/doc/source/getting_started/using_docker_swarm.rst index cecbeac20a..fc3fbb9fde 100644 --- a/doc/source/getting_started/using_docker_swarm.rst +++ b/doc/source/getting_started/using_docker_swarm.rst @@ -1,497 +1,41 @@ .. _using_docker_swarm: -======================================== -Distributed Deployment with Docker Swarm -======================================== +Docker Swarm: Deployment Considerations +======================================= -This document covers three topics: +This is a community experience report and architecture discussion, not a ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm setup. -1. Why Docker Swarm is a better choice than Kubernetes for a cluster of a dozen or so machines -2. Understanding Swarm by comparing it with Xinference's supervisor/worker -3. How to adapt the official compose file for Swarm, with an introduction to basic Swarm concepts and commands - -Tested with Xinference 2.2.0, Docker 29.2.1, Ubuntu 22.04, and NVIDIA driver 570 on 3 machines, each with one NVIDIA L20. The adjustments for Xinference 3.x in Section 4.8 are based on the official documentation and source code and have not yet been verified in a deployment. - -.. note:: - - Both Swarm and Xinference have a role called "worker", but the term refers to different things. In this document, "Swarm node" means a physical machine that has joined the Swarm cluster, and "Xinference worker" means the Xinference process that runs inference. - -1. Why Swarm is a better choice than Kubernetes for a cluster of a dozen or so machines -================================================= - -Our setup is a dozen or so GPU servers, maintained by a small team, mainly running Xinference. What we need from an orchestration tool: - -- Deploy containers across multiple machines -- Restart containers automatically when they exit abnormally -- Let containers on different machines reach each other -- Assign Xinference workers according to the number of GPUs -- Upgrade versions across all machines in one step - -Swarm handles all of these well, although GPU allocation requires extra manual configuration (see Section 1.1). - -Advantages of Swarm: - -- It is built into Docker, so there is nothing extra to install. ``docker swarm init`` creates a cluster and ``docker swarm join`` adds a machine -- The deployment file is a docker-compose file plus a ``deploy`` section, so anyone who can write compose files has little new to learn -- Deployment is a single command: ``docker stack deploy -c docker-compose-swarm.yml xinference`` - -What Kubernetes adds on top, such as load-based autoscaling, multi-tenant access control, and the Helm and Operator ecosystem, is not needed at this scale. In exchange, it brings learning and maintenance costs: etcd, the API server, the scheduler, network plugins, and GPU device plugins each need someone who understands them, and each of them can fail. For a small team, Kubernetes is overkill. - -Swarm does have limitations to be aware of: - -- **GPU support is Swarm's main weakness.** All of our machines have GPUs, so this directly affects deployment and operations. It is covered separately in Section 1.1 -- We use a single manager. While the manager machine is down, services cannot be deployed, upgraded, or adjusted; containers that are already running are not affected -- There are fewer community resources than for Kubernetes, so less help is available when problems come up - -If you later need to share GPUs among multiple teams or autoscale based on load, or if your organization already runs a Kubernetes cluster, Xinference also documents a Kubernetes deployment (see :ref:`using_kubernetes`), and you can migrate then. - -1.1 GPU Support: What to Know Before Choosing Swarm ---------------------------------------------------- - -**Swarm does not know about GPUs** - -Kubernetes itself does not know about GPUs either, but the agent Kubernetes runs on each node (kubelet) defines a gRPC interface called the device plugin interface. Any vendor can write a program that implements this interface to report which devices a node has and whether each one is healthy, and to decide which device a container gets when it starts. NVIDIA implements this interface in the NVIDIA device plugin. Once deployed on each node, the plugin detects the node's GPUs automatically, reports them, and marks a GPU as unavailable when it fails. The plugin itself has to be installed, and each node still needs the driver and the NVIDIA Container Toolkit. - -Swarm has no such interface. It only has a general-purpose counting mechanism called generic resources: - -1. When Docker starts, it reads ``node-generic-resources`` from ``/etc/docker/daemon.json``, written there by hand, and reports the values to the manager. The values do not change after that. Swarm does not know what these values represent; it treats them as a set of strings that can be allocated -2. At deployment time, Swarm picks one of the values for a container and passes it in through the environment variable ``DOCKER_RESOURCE_NVIDIA-GPU=`` -3. For the container to actually see only that GPU, nvidia-container-runtime has to read this environment variable. This requires setting ``swarm-resource = "DOCKER_RESOURCE_GPU"`` in ``/etc/nvidia-container-runtime/config.toml`` - -Swarm and NVIDIA work together through an agreed-upon environment variable name, not through a formal plugin mechanism. - -**Comparison** - -.. list-table:: - :header-rows: 1 - - * - - - Kubernetes + NVIDIA device plugin - - Swarm - * - GPU discovery - - Detected automatically by the plugin - - Written by hand into ``daemon.json`` on each machine - * - Configuration needed - - Install the plugin (or have the NVIDIA GPU Operator install it) - - Edit ``daemon.json`` and ``config.toml`` on each machine, then restart Docker - * - GPU failure - - The plugin marks the GPU unhealthy and stops assigning it - - Swarm is unaware and keeps assigning containers to it - * - Replacing a GPU - - Updated automatically by the plugin - - The UUID changes; reconfigure and restart Docker - * - Identification - - Handled by the plugin - - GPU UUIDs must be entered by hand; GPU indices do not work - -**What this means in practice** - -- Every machine needs two extra configuration steps before deployment. If they are skipped or done wrong, the Xinference worker stays Pending with ``insufficient resources``. We ran into this during deployment: declaring the GPU by index (``NVIDIA-GPU=0``) caused this error, removing ``node-generic-resources`` caused the same error, and it only worked after switching to GPU UUIDs -- These two steps must not be skipped when adding a machine; otherwise the worker does not start after ``docker swarm join`` -- After a GPU is replaced, the configuration must be redone -- Swarm does not notice when a GPU fails. You have to detect the problem yourself through ``nvidia-smi`` or the Xinference Web UI, then take that machine out of the cluster (see "GPU failure" in the deployment guide) -- The number of GPUs per machine determines how Xinference workers are laid out. With one GPU per machine, ``mode: global`` is the simplest; machines with multiple GPUs need a different approach (see "Multi-GPU machines" in the deployment guide) - -We wrote a script, ``setup-gpu-swarm.sh``, that gets the GPU UUIDs and edits both configuration files automatically. It reduces manual work, but it still has to be run once on every machine, and again after a GPU is replaced. - -**Why we still chose Swarm** - -Our cluster has a dozen or so machines with identical configurations, and GPUs are rarely replaced. The manual configuration is a one-time task, and with the script it takes a few minutes per machine. Compared with deploying and maintaining a Kubernetes cluster, this cost is acceptable. - -If the GPU models or counts in your cluster change often, or you need GPU failures to be handled automatically, Kubernetes has a clear advantage here. - -2. Understanding Swarm Through Xinference's Supervisor/Worker -============================================================= - -2.1 Similar Structure, Different Responsibilities -------------------------------------------------- - -Both Swarm and Xinference follow a "one control node + multiple execution nodes" structure. If you are familiar with Xinference, you can use its supervisor/worker to understand Swarm's manager/node: - -.. list-table:: - :header-rows: 1 - - * - - - Swarm - - Xinference - * - Control role - - manager - - supervisor - * - Execution role - - Swarm node (physical machine) - - Xinference worker (process in a container) - * - What it schedules - - Containers - - Models - * - Resources it tracks - - CPU, memory, and GPUs of each machine - - GPU memory and model usage - -Swarm decides which machine a container runs on, without knowing what runs inside the container. Xinference decides which worker loads a model and where inference requests go, but it does not start worker processes. In our deployment, these two layers work on top of each other. - -The two layers meet at the GPU: Swarm assigns a GPU to a Xinference worker container, and the container can see only that GPU; the Xinference worker detects that GPU at startup and loads models on it. - -2.2 The Key Difference: The Supervisor Is Passive, the Manager Is Active ------------------------------------------------------------------------- - -The similar structure raises a common question: if Xinference already has supervisor/worker, why use Swarm? - -The answer lies in how the two work. - -**Xinference's supervisor is passive.** It only accepts registrations: the worker process on each machine connects to the supervisor after starting, and only then does the supervisor know about that worker. The supervisor cannot start a worker on another machine. When a worker process exits, the supervisor can only remove it from its list; it cannot bring it back up. - -**Swarm's manager is active.** It makes sure these processes exist: which machine to start each container on, with which image and arguments, and restarting containers when they fail. - -So without Swarm, Xinference's supervisor/worker still works, but someone has to do the following by hand: - -- Log in to each machine and run ``docker run`` with the correct supervisor address, local IP, port, and other arguments -- When upgrading Xinference, stop the container, pull the image, and restart it on each machine in turn -- When adding a machine, deploy a worker on it manually -- Log in to each machine to check status and logs -- When the machine running the supervisor goes down, restart the supervisor on another machine manually - -With Swarm: - -- Run ``docker stack deploy`` once on the manager, and the supervisor and workers on all machines are deployed together -- When a container exits abnormally, Swarm restarts it automatically -- After a new machine runs ``docker swarm join``, a Xinference worker is deployed on it automatically -- To upgrade, change the image version in the compose file and run ``docker stack deploy`` again -- Use ``docker service ps`` and ``docker service logs`` on the manager to check status and logs across all machines +Why consider Swarm? +------------------- -In short: Xinference manages models, and Swarm manages containers. +For a small team managing a dozen or so similar GPU servers, Swarm can reduce the manual work of deploying, restarting and updating containers across machines. It is built into Docker Engine. This was the contributor's reason for choosing it, not a rule that small clusters should always prefer Swarm. -2.3 What Swarm Does Not Solve ------------------------------ +Consider existing operational expertise, GPU management and availability requirements as well as cluster size. Teams already operating Kubernetes or needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`. -- When the machine running the supervisor goes down, Swarm restarts the supervisor on another machine, and Xinference workers can still find it through the service name. However, the new supervisor does not keep records of previously loaded models, and we have not verified whether Xinference workers re-register automatically -- If the machine that goes down happens to be the manager, Swarm itself cannot reschedule anything +Two layers of scheduling +------------------------ -With only two or three machines, manual maintenance is not costly. The more machines there are, the more manual work Swarm saves. +Swarm managers schedule container tasks on Docker nodes and maintain the services' desired state. Managers can also run tasks. A Xinference supervisor manages model placement and requests; Xinference worker processes register with it and run models. The supervisor does not provision worker containers: container orchestration and model serving are separate responsibilities. -3. Swarm Basics -=============== +A global Swarm service runs one task per eligible node, not one per GPU. GPU reservations and placement constraints must match the intended worker layout. Internal actor addresses must identify reachable individual processes; a load-balanced service address is not a substitute for each worker's address. -3.1 Node Roles +GPU management -------------- -A Swarm cluster has two kinds of machines: - -- **manager**: the machine that runs ``docker swarm init``. It handles scheduling and management. All commands for deploying and inspecting services are run on the manager -- **Regular node**: a machine that joins with ``docker swarm join``. It runs containers - -The manager also runs containers, just like regular nodes. - -Swarm supports multiple managers that back each other up (3 managers can tolerate the failure of 1); we use only one for simplicity. The token used to join decides the role: ``docker swarm join-token worker`` generates the token for a regular node, and ``docker swarm join-token manager`` generates the token for a manager. - -3.2 Key Concepts ----------------- - -.. list-table:: - :header-rows: 1 - - * - Concept - - Description - * - Stack - - A group of services deployed from one compose file with ``docker stack deploy``. Our stack is named ``xinference`` - * - Service - - Each service in the compose file, such as ``xinference-supervisor``. After deployment, the stack name is added as a prefix: ``xinference_xinference-supervisor`` - * - Task - - A running instance of a service. Normally each replica is one task. When a container fails and restarts, the old task is marked Failed and Swarm creates a new task to replace it. Each line in the output of ``docker service ps`` is a task - * - Container - - The container that a task actually runs on a machine - * - Overlay network - - A virtual network that spans machines; containers on different machines reach each other by service name - * - Ingress - - An overlay network that Swarm creates automatically to forward published ports to containers - -Hierarchy: Stack → Service → Task → Container - -The same replica slot may have several task records (the current one and earlier failed ones), but at any moment only one task is running, corresponding to one running container. - -You do not need to create or manage tasks, but you should be able to read the output of ``docker service ps`` when troubleshooting: - -.. code-block:: text - - ID NAME NODE DESIRED STATE CURRENT STATE ERROR - rdr5avu6kz8o xinference_xinference-worker.xx node-1 Running Running - v3lu6wcwdtuj \_ xinference_xinference-worker.xx node-1 Shutdown Failed "task: non-zero exit (1)" - -Lines with ``\_`` are earlier tasks in the same slot, and the ``NODE`` column shows the machine the container is on. - -3.3 Basic Commands ------------------- - -**Cluster management** - -.. code-block:: bash - - # On the manager: create the cluster - docker swarm init --advertise-addr - - # On the manager: show the command for joining the cluster - docker swarm join-token worker - - # On each other machine: join the cluster (copy the command from the previous step) - docker swarm join --token SWMTKN-1-xxxxx :2377 - - # On the manager: list all nodes - docker node ls - - # On a machine: leave the cluster - docker swarm leave - - # On the manager: remove the record of a node that has left - docker node rm - -``--advertise-addr`` tells other machines which IP to use to reach the manager. It can be omitted if the machine has only one network interface; with multiple network interfaces it is required, otherwise an error is reported. - -**Deploying and inspecting services** (run on the manager) - -.. code-block:: bash - - # Deploy or update (run again after editing the compose file) - docker stack deploy -c docker-compose-swarm.yml xinference - - # List all services and their replica counts - docker service ls - - # Show the tasks of a service: which machine and what state - docker service ps xinference_xinference-worker - - # Show full error messages (truncated by default) - docker service ps --no-trunc xinference_xinference-worker - - # Show logs, aggregated from the containers on all machines - docker service logs xinference_xinference-worker - docker service logs -f xinference_xinference-worker # follow - - # Remove the whole stack - docker stack rm xinference - -**Entering a container** +Swarm generic resources are manually advertised scheduling resources, not automatic GPU discovery or health monitoring. With the resource kind ``NVIDIA-GPU``, Swarm exports allocated UUIDs through ``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that same variable: ``swarm-resource = "DOCKER_RESOURCE_NVIDIA-GPU"``. Scheduling reservations alone do not enforce GPU visibility. -.. code-block:: bash +Operators must keep advertised GPU UUIDs current and handle failed GPUs. This manual work was acceptable for the contributor's stable, homogeneous machines, but it is an important trade-off when hardware changes frequently. See the `NVIDIA runtime implementation `_ for how the configured variable is selected. - # First, on the manager, use docker service ps to find the machine (NODE column) - # Then log in to that machine and run - docker ps - docker exec -it bash +Persistence and recovery +------------------------ -``docker stack``, ``docker service``, and ``docker node`` commands can only be run on the manager; ``docker ps`` and ``docker exec`` must be run on the machine where the container is running. +In current Xinference, disabling authentication does not make the supervisor stateless. Its default ``XINFERENCE_HOME`` contains launch history (including autostart configuration), monitoring configuration and system settings. Persist this state; persist authentication data and keys too when authentication is enabled. Environment variables for download settings do not replace these databases. -3.4 Compose Files Under Swarm ------------------------------ +A host bind mount stays on that host. Use placement on the specific storage host, or suitable durable storage available after rescheduling. A manager-role constraint alone does not select a unique host in a multi-manager cluster. Persist worker caches and engine environments separately. -The same compose file behaves differently when deployed with ``docker compose up`` and with ``docker stack deploy``: +Swarm can replace failed containers, but that does not establish Xinference model recovery or uninterrupted service. Test worker reconnection and model recovery for the chosen release. A single-manager Swarm cannot reschedule tasks while that manager is unavailable; surviving tasks can continue running. See `Swarm administration `_. -- ``depends_on`` is ignored. Swarm does not guarantee the start order of services, so applications have to retry on their own -- ``restart: always`` has no effect; use ``deploy.restart_policy`` instead -- ``build`` has no effect. Only prebuilt images can be used, and every machine must be able to pull them -- The ``deploy`` section only takes full effect under Swarm: replica count (``replicas``), one per machine (``mode: global``), resource reservations, placement constraints, and so on are all set here -- Configuration files are distributed with ``configs``. Swarm sends the file contents to the machine where the container runs, so you do not need a copy on every machine - -4. Adapting the Official Compose File -===================================== - -4.1 Starting Point ------------------- - -The starting point is `docker-compose-distributed.yml `__ in the Xinference repository. - -This file defines one supervisor and two workers. ``docker compose`` runs on a single machine only, so the file actually demonstrates the distributed structure on one machine. The comments in the file also say it is an example with two workers, and more can be added by incrementing the numbering. - -The repository also has a ``docker-compose.yml``. It is for standalone deployment: there is only one container, and the supervisor and worker run in the same process (``xinference-local``). It is not a suitable basis for a distributed deployment. - -When Xinference 3.0 was released, the standalone ``docker-compose.yml`` was updated, but the distributed file was not (its health check still uses curl and its image is still ``latest``). So the adjustments needed for 3.x are based on the standalone file and the 3.0 migration guide. - -4.2 The Supervisor Does Not Need a GPU --------------------------------------- - -The original file uses a YAML anchor that makes the supervisor inherit the GPU configuration: - -.. code-block:: yaml - - xinference: &xinference - deploy: - resources: - reservations: - devices: - - capabilities: [gpu] - - xinference-supervisor: - <<: *xinference # inherits the GPU configuration - -The supervisor only schedules; it does not run inference. The source code confirms this: the supervisor's startup code creates only a scheduling actor and an HTTP service, with no GPU-related code, while the worker's startup code detects the number of GPUs and passes the GPU list to the worker. This is a side effect of reusing the template. After the change, the supervisor does not request a GPU. - -4.3 Merging the Workers into One Service ----------------------------------------- - -The original file defines each worker as a separate service (``xinference-worker-1``, ``xinference-worker-2``), each with its own port. After the change, there is a single ``xinference-worker`` service with ``mode: global``, so Swarm runs one on every machine. When a new machine joins the cluster, a worker is deployed on it automatically, without editing the compose file. - -``mode: global`` suits machines with one GPU each. If machines have multiple GPUs, the worker layout needs to be reconsidered; see "Multi-GPU machines" in the deployment guide. - -4.4 Changing ``--host`` to ``$(hostname -i)`` ---------------------------------------------- - -This is the most important change. First, look at how Xinference registration works. - -When a worker starts, it sends its own address (``--host`` plus ``--worker-port``) to the supervisor: - -.. code-block:: python - - # worker side - await self._supervisor_ref.add_worker(self.address) - -When the supervisor receives it, it uses this address to connect back to the worker: - -.. code-block:: python - - # supervisor side - async def add_worker(self, worker_address: str): - worker_ref = await xo.actor_ref(address=worker_address, uid=WorkerActor.default_uid()) - -The same applies in the other direction: at startup, the worker first asks the supervisor for its internal address through the REST API, then connects to the supervisor using that address. - -So the value of ``--host`` **must be an address the other side can connect back to**. - -**Why the original file can use hostnames on a single machine** - -.. code-block:: bash - - xinference-worker -H xinference-worker-1 --worker-port 30001 - xinference-worker -H xinference-worker-2 --worker-port 30002 - -In a single-machine compose deployment, each service is defined separately and the service name is the hostname, which Docker DNS resolves to a unique container IP. All containers are on the same network and can reach each other. - -**Neither condition holds under Swarm** - -Once the workers become one service, all replicas share the service name ``xinference-worker`` and have no hostnames of their own. Swarm DNS offers only two kinds of resolution: ``xinference-worker`` resolves to a virtual IP that Swarm load-balances to one of the replicas, and ``tasks.xinference-worker`` returns the IPs of all replicas. Neither can point to a specific replica. - -If ``-H 0.0.0.0`` is used instead, every worker registers the address ``0.0.0.0:30001``, and when the supervisor connects back to that address, it connects to itself. - -The supervisor has the same problem. With ``--host 0.0.0.0``, the internal address the supervisor gives to workers is ``0.0.0.0:9999``, and a worker connecting to it connects to itself. This is the log from an actual deployment: - -.. code-block:: text - - xinference.core.worker ERROR Failed to upload node info: [Errno 111] Connect call failed ('0.0.0.0', 9999) - -At that point, the cluster information page of the Web UI showed 0 workers. - -**Solution** - -Inside the container, use ``hostname -i`` to get the container's IP on the overlay network, and start with that IP: - -.. code-block:: yaml - - command: sh -c 'xinference-supervisor --host $$(hostname -i) --port 9997 --supervisor-port 9999' - command: sh -c 'xinference-worker -e http://xinference-supervisor:9997 --host $$(hostname -i) --worker-port 30001' - -Each container has a unique IP on the overlay network that the others can reach, with no dependence on DNS. ``$$`` is an escape in compose files; the shell receives ``$(hostname -i)``. - -After the change, the supervisor address is an overlay IP such as ``10.0.1.3:9999``, and the 3 workers register as ``10.0.1.6:30001``, ``10.0.1.7:30001``, and ``10.0.1.8:30001``. - -4.5 Why nginx Is Needed ------------------------ - -After ``--host`` is changed to the overlay IP, workers can register, but the Web UI can no longer be reached from outside. The reason is that ``--host`` controls two things at once: - -- **The REST API listening address** (port 9997): the Web UI and HTTP API, which must be reachable from outside -- **The actor address** (port 9999): internal communication between the supervisor and workers, which must give workers an address they can connect back to - -.. code-block:: python - - # supervisor startup code - supervisor_address = f"{host}:{supervisor_port}" # actor address - restful_api.run(supervisor_address=supervisor_address, host=host, port=port) # the REST API uses the same host - -The two have conflicting requirements for the address: - -- ``--host 0.0.0.0``: the REST API listens on all network interfaces and is reachable from outside, but the actor address becomes ``0.0.0.0:9999`` and worker registration fails -- ``--host ``: worker registration works, but the REST API listens only on the overlay network - -Why does listening only on the overlay network make it unreachable from outside? In Swarm, a container that publishes a port has two network interfaces: one on the ingress network and one on the overlay network we defined (``xinference-net``). External traffic to ``:9997`` enters the container through the ingress network, but the REST API listens only on the overlay IP, so it never receives that traffic. - -A single-machine compose deployment does not have this problem: the container has only one network interface, so the port forwarding target and the address the service listens on are the same IP. - -Xinference's command line has no separate option for the REST API listening address (as of 3.5.0, the supervisor's only options are ``--host``, ``--port``, ``--supervisor-port``, and ``--log-level``), so an nginx container is added to forward traffic: - -- The supervisor does not publish a port and starts with its overlay IP, so internal communication works -- nginx publishes port 9997, listens on all network interfaces, and receives external traffic -- nginx forwards requests over the overlay network to the supervisor, using the service name ``xinference-supervisor`` - -.. code-block:: text - - Browser → any machine IP:9997 → ingress network → nginx container → overlay network → supervisor (10.0.1.x:9997) - -Ports published by Swarm are reachable on every machine in the cluster, so the Web UI can be opened through port 9997 on any machine's IP. - -If Xinference later provides separate options for the REST API listening address and the actor address, nginx can be removed. - -4.6 Requesting GPUs -------------------- - -The background for this change is in Section 1.1: Swarm does not know about GPUs, so GPUs can only be declared and allocated by hand through generic resources. - -On a single Docker host, GPUs can be used directly once the NVIDIA Container Toolkit is installed, and the original file uses this single-host syntax: - -.. code-block:: yaml - - # Original file: single-host docker compose syntax - deploy: - resources: - reservations: - devices: - - capabilities: [gpu] - driver: nvidia - count: all - -A single host involves no scheduling; Docker mounts the local GPUs into the container directly. Swarm has to decide which machine each container goes on, so it needs to know how many GPUs each machine has and how many are already assigned. That is why the syntax changes to generic resources: - -.. code-block:: yaml - - # After the change: Swarm syntax - deploy: - resources: - reservations: - generic_resources: - - discrete_resource_spec: - kind: 'NVIDIA-GPU' - value: 1 # each worker requests 1 GPU - -``kind`` must match the name used in ``node-generic-resources`` in each machine's ``daemon.json`` (``NVIDIA-GPU``). When scheduling, Swarm matches the number of GPUs declared on each machine against the number each worker requests; machines with no GPUs left are not assigned a worker. - -The compose file is only half of this mechanism. The other half is in ``daemon.json`` and ``config.toml`` on each machine, and both halves must be configured for it to work. - -4.7 Start Order +Further reading --------------- -Swarm ignores ``depends_on``, so the supervisor and workers start at almost the same time. If a worker starts before the supervisor is ready, the connection fails and the container exits. - -This is handled by the restart policy: the worker's ``restart_policy.delay`` is set to 20 seconds, so after a failure it waits 20 seconds before restarting, by which time the supervisor is up. In our tests, with 10 seconds each worker failed four times before succeeding; with 20 seconds it succeeded on the first try. - -4.8 Adjustments for Xinference 3.x ----------------------------------- - -Several changes in Xinference 3.0 affect deployment. The following adjustments were made on 3.5.0: - -.. list-table:: - :header-rows: 1 - - * - Change - - Adjustment - * - The official GPU image is based on CUDA 13 and requires NVIDIA driver 580 or later - - Upgrade the driver on all machines - * - Authentication is enabled by default; every API call requires a login or an API key - - For use on an internal network, disable it with ``XINFERENCE_AUTH_ADVANCED=false`` - * - The image no longer bundles inference engines such as vLLM and SGLang; they are installed into ``XINFERENCE_HOME/virtualenv`` the first time a model is launched - - Mount the worker's ``/root/.xinference`` to a host directory so engines are not reinstalled when the container is recreated; configure a pip mirror reachable from your network - * - The official docs state that the image is guaranteed to contain python3, but not curl - - Use python3 for the health check - * - The Web UI moved from ``/ui`` to ``/`` - - Open the Web UI at ``http://:9997/`` - * - The official recommendation is to pin the image version in production - - Use ``xprobe/xinference:v3.5.0`` instead of ``latest``, so machines do not pull different versions at different times - -With authentication disabled, the only data the supervisor needs to persist is ``XINFERENCE_HOME/system-settings.json``, which holds model download settings (download source, pip index, and so on). If these settings are changed in the Web UI, they are lost when the supervisor is rescheduled to another machine. So keep these settings in environment variables in the compose file instead of changing them in the Web UI. That way, the supervisor needs no mounted directory and does not have to be pinned to a particular machine. - -If you later enable authentication, the authentication database and encryption key are stored under the supervisor's ``XINFERENCE_HOME/auth``. In that case, pin the supervisor to the manager (``placement.constraints: [node.role == manager]``, a built-in Swarm attribute that requires no labels) and mount ``/root/.xinference`` to a host directory. +See `Swarm concepts `_ for the orchestration model, :ref:`using_docker_compose` for the existing Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A complete Swarm configuration and current-version cluster validation are outside the scope of this report. diff --git a/doc/source/locale/de/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/de/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..92f32edaa7 --- /dev/null +++ b/doc/source/locale/de/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,185 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: de\n" +"Language: de\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm: Überlegungen zur Bereitstellung" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"Dies ist ein Erfahrungsbericht aus der Community mit einer " +"Architekturdiskussion, keine direkt ausführbare Bereitstellungsanleitung. " +"Der Beitragende testete Xinference 2.2.0 mit Docker 29.2.1, Ubuntu 22.04 und " +"NVIDIA-Treiber 570 auf drei Rechnern mit jeweils einer NVIDIA L20. " +"Xinference 3.x wurde in diesem Swarm-Aufbau nicht validiert." + +msgid "Why consider Swarm?" +msgstr "Warum Swarm in Betracht ziehen?" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"Für ein kleines Team mit etwa einem Dutzend ähnlicher GPU-Server kann Swarm " +"den manuellen Aufwand für Bereitstellung, Neustart und Aktualisierung von " +"Containern auf mehreren Rechnern reduzieren. Es ist in Docker Engine " +"integriert. Dies war der Grund für die Wahl des Beitragenden, keine " +"allgemeine Empfehlung für alle kleinen Cluster." + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"Berücksichtigen Sie neben der Clustergröße auch Betriebserfahrung, GPU-" +"Verwaltung und Verfügbarkeitsanforderungen. Teams, die bereits Kubernetes " +"betreiben oder dessen Geräte-Plugin-Ökosystem benötigen, bevorzugen " +"möglicherweise :ref:`using_kubernetes`." + +msgid "Two layers of scheduling" +msgstr "Zwei Planungsebenen" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"Swarm-Manager planen Container-Tasks auf Docker-Knoten und erhalten den " +"gewünschten Zustand der Dienste. Manager können selbst Tasks ausführen. Ein " +"Xinference-Supervisor verwaltet Modellplatzierung und Anfragen; Xinference-" +"Worker-Prozesse registrieren sich dort und führen Modelle aus. Der " +"Supervisor stellt keine Worker-Container bereit: Container-Orchestrierung " +"und Modellbereitstellung sind getrennte Aufgaben." + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Ein globaler Swarm-Dienst führt einen Task pro geeignetem Knoten aus, nicht " +"pro GPU. GPU-Reservierungen und Platzierungsbedingungen müssen zur " +"vorgesehenen Worker-Anordnung passen. Interne Actor-Adressen müssen " +"erreichbare einzelne Prozesse identifizieren; eine lastverteilte " +"Dienstadresse ersetzt nicht die Adresse jedes Workers." + +msgid "GPU management" +msgstr "GPU-Verwaltung" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Generische Swarm-Ressourcen sind manuell deklarierte Planungsressourcen, " +"keine automatische GPU-Erkennung oder Zustandsüberwachung. Beim " +"Ressourcentyp ``NVIDIA-GPU`` exportiert Swarm zugewiesene UUIDs über " +"``DOCKER_RESOURCE_NVIDIA-GPU``. Die NVIDIA-Runtime muss dieselbe Variable " +"lesen: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. Reservierungen " +"allein erzwingen keine Einschränkung der GPU-Sichtbarkeit." + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"Betreiber müssen die deklarierten GPU-UUIDs aktuell halten und ausgefallene " +"GPUs behandeln. Für die stabilen, homogenen Rechner des Beitragenden war " +"dieser manuelle Aufwand akzeptabel; bei häufig wechselnder Hardware ist er " +"jedoch ein wichtiger Kompromiss. Die Auswahl der konfigurierten Variable " +"zeigt die `NVIDIA-Runtime-Implementierung `_." + +msgid "Persistence and recovery" +msgstr "Persistenz und Wiederherstellung" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"Im aktuellen Xinference wird der Supervisor durch das Abschalten der " +"Authentifizierung nicht zustandslos. Sein standardmäßiges " +"``XINFERENCE_HOME`` enthält den Startverlauf (einschließlich Autostart-" +"Konfiguration), Überwachungskonfiguration und Systemeinstellungen. Speichern " +"Sie diesen Zustand dauerhaft; bei aktivierter Authentifizierung auch " +"Authentifizierungsdaten und Schlüssel. Umgebungsvariablen für Download-" +"Einstellungen ersetzen diese Datenbanken nicht." + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"Die Daten eines Host-Bind-Mounts bleiben auf diesem Host. Binden Sie die " +"Platzierung an den konkreten Speicherhost oder nutzen Sie geeigneten " +"dauerhaften Speicher, der nach einer Neuplatzierung erreichbar bleibt. Eine " +"reine Manager-Rollenbedingung wählt in einem Cluster mit mehreren Managern " +"keinen eindeutigen Host. Worker-Caches und Engine-Umgebungen sind separat zu " +"persistieren." + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm kann ausgefallene Container ersetzen, garantiert damit aber weder die " +"Wiederherstellung von Xinference-Modellen noch einen unterbrechungsfreien " +"Dienst. Testen Sie Worker-Wiederverbindung und Modellwiederherstellung für " +"die gewählte Version. Bei nur einem Manager kann Swarm während dessen " +"Ausfall keine Tasks neu platzieren; verbleibende Tasks können weiterlaufen. " +"Siehe `Swarm-Verwaltung `_." + +msgid "Further reading" +msgstr "Weiterführende Informationen" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"Siehe `Swarm-Konzepte `_ " +"zum Orchestrierungsmodell, :ref:`using_docker_compose` zur vorhandenen " +"Compose-Konfiguration und :doc:`migration_3_0` zu Änderungen in Xinference " +"3.x. Eine vollständige Swarm-Konfiguration und Cluster-Validierung der " +"aktuellen Version liegen außerhalb dieses Berichts." diff --git a/doc/source/locale/es/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/es/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..8d348f41e9 --- /dev/null +++ b/doc/source/locale/es/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,186 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: es\n" +"Language: es\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm: consideraciones de despliegue" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"Este documento es un informe de experiencia de la comunidad y una discusión " +"de arquitectura, no una guía de despliegue lista para ejecutar. El " +"colaborador probó Xinference 2.2.0 con Docker 29.2.1, Ubuntu 22.04 y el " +"controlador NVIDIA 570 en tres máquinas, cada una con una NVIDIA L20. " +"Xinference 3.x no se ha validado en esta configuración Swarm." + +msgid "Why consider Swarm?" +msgstr "¿Por qué considerar Swarm?" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"Para un equipo pequeño que administra una docena de servidores GPU " +"similares, Swarm puede reducir el trabajo manual de desplegar, reiniciar y " +"actualizar contenedores en varias máquinas. Está integrado en Docker Engine. " +"Este fue el motivo del colaborador para elegirlo, no una regla que obligue a " +"los clústeres pequeños a preferir Swarm." + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"Considere la experiencia operativa, la gestión de GPU y los requisitos de " +"disponibilidad además del tamaño del clúster. Los equipos que ya operan " +"Kubernetes o necesitan su ecosistema de complementos de dispositivos pueden " +"preferir :ref:`using_kubernetes`." + +msgid "Two layers of scheduling" +msgstr "Dos capas de planificación" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"Los managers de Swarm asignan tareas de contenedores a nodos Docker y " +"mantienen el estado deseado de los servicios. También pueden ejecutar " +"tareas. Un supervisor de Xinference gestiona la ubicación de modelos y las " +"solicitudes; los procesos worker de Xinference se registran en él y ejecutan " +"modelos. El supervisor no crea contenedores worker: la orquestación de " +"contenedores y el servicio de modelos son responsabilidades distintas." + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Un servicio global de Swarm ejecuta una tarea por nodo apto, no por GPU. Las " +"reservas de GPU y las restricciones de ubicación deben corresponder a la " +"distribución prevista de workers. Las direcciones internas de los actors " +"deben identificar procesos individuales accesibles; una dirección de " +"servicio con balanceo de carga no sustituye a la dirección de cada worker." + +msgid "GPU management" +msgstr "Gestión de GPU" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Los recursos genéricos de Swarm se declaran manualmente para la " +"planificación; no ofrecen descubrimiento automático ni supervisión del " +"estado de las GPU. Con el tipo ``NVIDIA-GPU``, Swarm exporta los UUID " +"asignados mediante ``DOCKER_RESOURCE_NVIDIA-GPU``. El runtime de NVIDIA debe " +"leer esa misma variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-" +"GPU\"``. Las reservas de planificación por sí solas no restringen la " +"visibilidad de GPU." + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"Los operadores deben mantener actualizados los UUID declarados y gestionar " +"las GPU averiadas. Este trabajo manual era aceptable para las máquinas " +"estables y homogéneas del colaborador, pero es un compromiso importante " +"cuando el hardware cambia a menudo. Consulte la `implementación del runtime " +"de NVIDIA `_ para conocer cómo se " +"selecciona la variable configurada." + +msgid "Persistence and recovery" +msgstr "Persistencia y recuperación" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"En Xinference actual, desactivar la autenticación no convierte al supervisor " +"en un servicio sin estado. Su ``XINFERENCE_HOME`` predeterminado contiene el " +"historial de lanzamientos (incluida la configuración de inicio automático), " +"la configuración de monitorización y los ajustes del sistema. Haga " +"persistente este estado, así como los datos de autenticación y las claves " +"cuando se active la autenticación. Las variables de entorno de descarga no " +"sustituyen estas bases de datos." + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"Los datos de un montaje vinculado al host permanecen en ese host. Fije la " +"ubicación al host de almacenamiento concreto o use almacenamiento duradero " +"adecuado que siga disponible tras la reubicación. Una restricción del rol " +"manager no selecciona un host único en un clúster con varios managers. Haga " +"persistentes por separado las cachés y los entornos de motores de los " +"workers." + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm puede sustituir contenedores fallidos, pero esto no garantiza la " +"recuperación de modelos de Xinference ni un servicio ininterrumpido. Pruebe " +"la reconexión de workers y la recuperación de modelos con la versión " +"elegida. Un Swarm con un solo manager no puede reubicar tareas mientras ese " +"manager no esté disponible; las tareas supervivientes pueden seguir " +"ejecutándose. Consulte la `administración de Swarm `_." + +msgid "Further reading" +msgstr "Lecturas adicionales" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"Consulte los `conceptos de Swarm `_ para el modelo de orquestación, :ref:`using_docker_compose` " +"para la configuración Compose existente y :doc:`migration_3_0` para los " +"cambios de Xinference 3.x. Una configuración Swarm completa y la validación " +"en clúster de la versión actual quedan fuera del alcance de este informe." diff --git a/doc/source/locale/fr/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/fr/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..02ce8b024b --- /dev/null +++ b/doc/source/locale/fr/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,187 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: fr\n" +"Language: fr\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm : considérations de déploiement" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"Ce document présente une expérience communautaire et une discussion " +"d'architecture, pas un guide de déploiement prêt à exécuter. Le contributeur " +"a testé Xinference 2.2.0 avec Docker 29.2.1, Ubuntu 22.04 et le pilote " +"NVIDIA 570 sur trois machines équipées chacune d'une NVIDIA L20. Xinference " +"3.x n'a pas été validé dans cette configuration Swarm." + +msgid "Why consider Swarm?" +msgstr "Pourquoi envisager Swarm ?" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"Pour une petite équipe gérant une douzaine de serveurs GPU similaires, Swarm " +"peut réduire le travail manuel de déploiement, de redémarrage et de mise à " +"jour des conteneurs sur plusieurs machines. Il est intégré à Docker Engine. " +"C'est la raison du choix du contributeur, et non une règle imposant Swarm " +"aux petits clusters." + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"Tenez compte de l'expérience d'exploitation, de la gestion des GPU et des " +"exigences de disponibilité, pas seulement de la taille du cluster. Les " +"équipes utilisant déjà Kubernetes ou ayant besoin de son écosystème de " +"plugins de périphériques peuvent préférer :ref:`using_kubernetes`." + +msgid "Two layers of scheduling" +msgstr "Deux niveaux d'ordonnancement" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"Les managers Swarm placent les tâches de conteneurs sur les nœuds Docker et " +"maintiennent l'état souhaité des services. Ils peuvent aussi exécuter des " +"tâches. Le supervisor Xinference gère le placement des modèles et les " +"requêtes ; les processus worker Xinference s'y enregistrent et exécutent les " +"modèles. Le supervisor ne crée pas les conteneurs worker : l'orchestration " +"des conteneurs et le service des modèles sont des responsabilités distinctes." + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Un service global Swarm exécute une tâche par nœud admissible, et non par " +"GPU. Les réservations GPU et les contraintes de placement doivent " +"correspondre à l'organisation prévue des workers. Les adresses internes des " +"actors doivent identifier des processus individuels accessibles ; une " +"adresse de service avec équilibrage de charge ne remplace pas l'adresse de " +"chaque worker." + +msgid "GPU management" +msgstr "Gestion des GPU" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Les ressources génériques Swarm sont des ressources d'ordonnancement " +"déclarées manuellement, sans découverte automatique ni surveillance de santé " +"des GPU. Pour le type ``NVIDIA-GPU``, Swarm exporte les UUID alloués via " +"``DOCKER_RESOURCE_NVIDIA-GPU``. Le runtime NVIDIA doit lire cette même " +"variable : ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. Les seules " +"réservations d'ordonnancement ne limitent pas la visibilité des GPU." + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"Les opérateurs doivent actualiser les UUID GPU déclarés et gérer les GPU " +"défaillants. Ce travail manuel était acceptable pour les machines stables et " +"homogènes du contributeur, mais constitue un compromis important lorsque le " +"matériel change souvent. Consultez l'`implémentation du runtime NVIDIA " +"`_ pour la sélection de la variable " +"configurée." + +msgid "Persistence and recovery" +msgstr "Persistance et reprise" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"Dans Xinference actuel, désactiver l'authentification ne rend pas le " +"supervisor sans état. Son ``XINFERENCE_HOME`` par défaut contient " +"l'historique des lancements (dont la configuration de démarrage " +"automatique), la configuration de surveillance et les paramètres système. " +"Rendez cet état persistant, ainsi que les données d'authentification et les " +"clés si l'authentification est activée. Les variables d'environnement des " +"paramètres de téléchargement ne remplacent pas ces bases de données." + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"Les données d'un montage lié à l'hôte restent sur cet hôte. Fixez le " +"placement sur l'hôte de stockage précis, ou utilisez un stockage durable " +"adapté et accessible après réordonnancement. Une contrainte de rôle manager " +"seule ne sélectionne pas un hôte unique dans un cluster à plusieurs " +"managers. Conservez séparément les caches et les environnements des moteurs " +"des workers." + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm peut remplacer les conteneurs défaillants, mais cela ne garantit ni la " +"reprise des modèles Xinference ni un service ininterrompu. Testez la " +"reconnexion des workers et la reprise des modèles pour la version choisie. " +"Avec un seul manager, Swarm ne peut pas réordonner les tâches lorsque celui-" +"ci est indisponible ; les tâches survivantes peuvent continuer. Voir " +"l'`administration de Swarm `_." + +msgid "Further reading" +msgstr "Pour aller plus loin" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"Consultez les `concepts de Swarm `_ pour le modèle d'orchestration, :ref:`using_docker_compose` " +"pour la configuration Compose existante et :doc:`migration_3_0` pour les " +"changements de Xinference 3.x. Une configuration Swarm complète et la " +"validation en cluster de la version actuelle sont hors du périmètre de ce " +"rapport." diff --git a/doc/source/locale/it/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/it/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..5ebeaf6730 --- /dev/null +++ b/doc/source/locale/it/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,187 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: it\n" +"Language: it\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm: considerazioni sul deployment" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"Questo documento è un resoconto di esperienza della comunità e una " +"discussione architetturale, non una guida al deployment pronta da eseguire. " +"Il contributore ha testato Xinference 2.2.0 con Docker 29.2.1, Ubuntu 22.04 " +"e il driver NVIDIA 570 su tre macchine, ciascuna con una NVIDIA L20. " +"Xinference 3.x non è stato validato in questa configurazione Swarm." + +msgid "Why consider Swarm?" +msgstr "Perché considerare Swarm?" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"Per un piccolo team che gestisce una dozzina di server GPU simili, Swarm può " +"ridurre il lavoro manuale di distribuzione, riavvio e aggiornamento dei " +"container su più macchine. È integrato in Docker Engine. Questo è il motivo " +"della scelta del contributore, non una regola secondo cui i piccoli cluster " +"dovrebbero sempre preferire Swarm." + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"Considerate l'esperienza operativa, la gestione delle GPU e i requisiti di " +"disponibilità oltre alle dimensioni del cluster. I team che già utilizzano " +"Kubernetes o necessitano del suo ecosistema di plugin per dispositivi " +"potrebbero preferire :ref:`using_kubernetes`." + +msgid "Two layers of scheduling" +msgstr "Due livelli di scheduling" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"I manager Swarm assegnano i task dei container ai nodi Docker e mantengono " +"lo stato desiderato dei servizi. Possono anche eseguire task. Un supervisor " +"Xinference gestisce il posizionamento dei modelli e le richieste; i processi " +"worker Xinference si registrano presso di esso ed eseguono i modelli. Il " +"supervisor non crea i container worker: l'orchestrazione dei container e il " +"servizio dei modelli sono responsabilità distinte." + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Un servizio globale Swarm esegue un task per nodo idoneo, non per GPU. Le " +"prenotazioni GPU e i vincoli di posizionamento devono corrispondere alla " +"disposizione prevista dei worker. Gli indirizzi interni degli actor devono " +"identificare singoli processi raggiungibili; un indirizzo di servizio con " +"bilanciamento del carico non sostituisce l'indirizzo di ciascun worker." + +msgid "GPU management" +msgstr "Gestione delle GPU" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Le risorse generiche Swarm sono risorse di scheduling dichiarate " +"manualmente, non un sistema di rilevamento automatico o monitoraggio dello " +"stato delle GPU. Con il tipo ``NVIDIA-GPU``, Swarm esporta gli UUID " +"assegnati tramite ``DOCKER_RESOURCE_NVIDIA-GPU``. Il runtime NVIDIA deve " +"leggere la stessa variabile: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-" +"GPU\"``. Le sole prenotazioni di scheduling non limitano la visibilità delle " +"GPU." + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"Gli operatori devono aggiornare gli UUID GPU dichiarati e gestire le GPU " +"guaste. Questo lavoro manuale era accettabile per le macchine stabili e " +"omogenee del contributore, ma è un compromesso importante quando l'hardware " +"cambia spesso. Per la selezione della variabile configurata, consultate " +"l'`implementazione del runtime NVIDIA `_." + +msgid "Persistence and recovery" +msgstr "Persistenza e ripristino" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"Nell'attuale Xinference, disabilitare l'autenticazione non rende il " +"supervisor privo di stato. Il suo ``XINFERENCE_HOME`` predefinito contiene " +"la cronologia degli avvii (inclusa la configurazione di avvio automatico), " +"la configurazione di monitoraggio e le impostazioni di sistema. Rendete " +"persistente questo stato, nonché i dati di autenticazione e le chiavi quando " +"l'autenticazione è abilitata. Le variabili d'ambiente per il download non " +"sostituiscono questi database." + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"I dati di un bind mount dell'host rimangono su quell'host. Vincolate il " +"posizionamento allo specifico host di archiviazione oppure usate uno storage " +"durevole appropriato, accessibile dopo il riposizionamento. Il solo vincolo " +"di ruolo manager non seleziona un host univoco in un cluster con più " +"manager. Rendete persistenti separatamente le cache e gli ambienti dei " +"motori dei worker." + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm può sostituire container guasti, ma questo non garantisce il " +"ripristino dei modelli Xinference o un servizio ininterrotto. Testate la " +"riconnessione dei worker e il ripristino dei modelli per la release scelta. " +"Un Swarm con un solo manager non può riposizionare i task mentre quel " +"manager non è disponibile; i task superstiti possono continuare. Consultate " +"l'`amministrazione di Swarm `_." + +msgid "Further reading" +msgstr "Approfondimenti" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"Consultate i `concetti di Swarm `_ per il modello di orchestrazione, :ref:`using_docker_compose` " +"per la configurazione Compose esistente e :doc:`migration_3_0` per le " +"modifiche di Xinference 3.x. Una configurazione Swarm completa e la " +"validazione in cluster della versione attuale non rientrano nell'ambito di " +"questo resoconto." diff --git a/doc/source/locale/ja/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/ja/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..2ebaa0c238 --- /dev/null +++ b/doc/source/locale/ja/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,172 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: ja\n" +"Language: ja\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm：デプロイ時の検討事項" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"本稿はコミュニティの実践報告とアーキテクチャの解説であり、そのまま実行できる" +"デプロイ手順ではありません。投稿者は Xinference 2.2.0、Docker 29.2.1、Ubuntu " +"22.04、NVIDIA ドライバー 570 を使用し、それぞれ NVIDIA L20 を1枚搭載した3台で" +"テストしました。この Swarm 構成で Xinference 3.x は未検証です。" + +msgid "Why consider Swarm?" +msgstr "Swarm を検討する理由" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"同様の構成の GPU サーバーを十数台管理する小規模チームでは、Swarm により複数マ" +"シンへのコンテナのデプロイ、再起動、更新の手作業を減らせます。Swarm は " +"Docker Engine に組み込まれています。これは投稿者の選択理由であり、小規模クラ" +"スターでは常に Swarm を選ぶべきという意味ではありません。" + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"クラスター規模だけでなく、既存の運用経験、GPU 管理、可用性要件も考慮してくだ" +"さい。すでに Kubernetes を運用している場合や、そのデバイスプラグインのエコシ" +"ステムが必要な場合は、:ref:`using_kubernetes` が適していることもあります。" + +msgid "Two layers of scheduling" +msgstr "2層のスケジューリング" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"Swarm manager は Docker ノードにコンテナタスクを配置し、サービスの望ましい状" +"態を維持します。manager 自身もタスクを実行できます。Xinference supervisor は" +"モデルの配置とリクエストを管理し、Xinference worker プロセスはそこに登録して" +"モデルを実行します。supervisor は worker コンテナを作成しません。コンテナの" +"オーケストレーションとモデル提供は別の責務です。" + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Swarm の global サービスは、条件を満たすノードごとに1タスクを実行し、GPU ごと" +"ではありません。GPU のリソース予約と配置制約は、意図する worker 構成に合わせ" +"る必要があります。内部 actor アドレスは到達可能な個々のプロセスを識別する必要" +"があり、負荷分散されるサービスアドレスは各 worker のアドレスの代わりにはなり" +"ません。" + +msgid "GPU management" +msgstr "GPU 管理" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Swarm の汎用リソースは手動で宣言するスケジューリング資源であり、GPU の自動検" +"出やヘルス監視ではありません。リソース種別が ``NVIDIA-GPU`` の場合、Swarm は" +"割り当てた UUID を ``DOCKER_RESOURCE_NVIDIA-GPU`` で渡します。NVIDIA runtime " +"も同じ変数を読むように ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"`` と" +"設定する必要があります。リソース予約だけでは GPU の可視性は制限されません。" + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"運用者は宣言した GPU UUID を最新に保ち、故障した GPU に対処する必要がありま" +"す。構成が安定した同種のマシンを使う投稿者には許容できる作業でしたが、ハード" +"ウェアが頻繁に変わる場合は重要なトレードオフです。変数の選択方法は `NVIDIA " +"runtime の実装 `_ を参照してください。" + +msgid "Persistence and recovery" +msgstr "永続化と復旧" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"現在の Xinference では、認証を無効にしても supervisor はステートレスになりま" +"せん。デフォルトの ``XINFERENCE_HOME`` には起動履歴（自動起動設定を含む）、監" +"視設定、システム設定があります。これらを永続化し、認証を有効にする場合は認証" +"データと鍵も永続化してください。ダウンロード設定の環境変数はこれらのデータ" +"ベースの代替にはなりません。" + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"ホストのバインドマウントのデータはそのホストに残ります。特定のストレージホス" +"トに配置を固定するか、再配置後も利用できる適切な永続ストレージを使用してくだ" +"さい。複数 manager のクラスターでは、manager ロールの制約だけではホストを一意" +"に選べません。worker のキャッシュとエンジン環境は別途永続化します。" + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm は故障したコンテナを置き換えられますが、Xinference のモデル復旧や無停止" +"サービスを保証するものではありません。使用するリリースで worker の再接続とモ" +"デル復旧をテストしてください。manager が1台の Swarm では、その manager が利用" +"できない間はタスクを再配置できませんが、生存しているタスクは実行を継続できま" +"す。`Swarm の管理 `_ を参" +"照してください。" + +msgid "Further reading" +msgstr "関連資料" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"オーケストレーションの仕組みは `Swarm の概念 `_、既存の Compose 構成は :ref:`using_docker_compose`、" +"Xinference 3.x の変更は :doc:`migration_3_0` を参照してください。完全な " +"Swarm 構成と現行バージョンのクラスター検証は本稿の対象外です。" diff --git a/doc/source/locale/ko/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/ko/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..d058c8beb5 --- /dev/null +++ b/doc/source/locale/ko/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,172 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: ko\n" +"Language: ko\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm: 배포 시 고려 사항" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"이 문서는 커뮤니티의 경험 보고와 아키텍처 설명이며, 바로 실행할 수 있는 배포 " +"가이드는 아닙니다. 기여자는 NVIDIA L20이 한 장씩 장착된 머신 세 대에서 " +"Xinference 2.2.0, Docker 29.2.1, Ubuntu 22.04, NVIDIA 드라이버 570을 테스트했" +"습니다. 이 Swarm 구성에서 Xinference 3.x는 아직 검증하지 않았습니다." + +msgid "Why consider Swarm?" +msgstr "Swarm을 고려하는 이유" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"비슷한 구성의 GPU 서버 십여 대를 관리하는 소규모 팀이라면 Swarm으로 여러 머신" +"의 컨테이너 배포, 재시작, 업데이트에 필요한 수작업을 줄일 수 있습니다. Swarm" +"은 Docker Engine에 내장되어 있습니다. 이는 기여자가 선택한 이유이지, 작은 클" +"러스터라면 항상 Swarm을 우선해야 한다는 뜻은 아닙니다." + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"클러스터 규모뿐 아니라 기존 운영 경험, GPU 관리, 가용성 요구 사항도 고려하세" +"요. 이미 Kubernetes를 운영하거나 그 디바이스 플러그인 생태계가 필요한 팀" +"은 :ref:`using_kubernetes` 를 선호할 수 있습니다." + +msgid "Two layers of scheduling" +msgstr "두 계층의 스케줄링" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"Swarm manager는 Docker 노드에 컨테이너 태스크를 배치하고 서비스의 원하는 상태" +"를 유지합니다. manager도 태스크를 실행할 수 있습니다. Xinference supervisor" +"는 모델 배치와 요청을 관리하며, Xinference worker 프로세스는 여기에 등록하고 " +"모델을 실행합니다. supervisor는 worker 컨테이너를 생성하지 않습니다. 컨테이" +"너 오케스트레이션과 모델 서빙은 별개의 책임입니다." + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Swarm의 global 서비스는 GPU마다가 아니라 조건을 충족하는 노드마다 태스크 하나" +"를 실행합니다. GPU 리소스 예약과 배치 제약 조건은 의도한 worker 구성에 맞아" +"야 합니다. 내부 actor 주소는 접근 가능한 개별 프로세스를 식별해야 하며, 로드 " +"밸런싱되는 서비스 주소로 각 worker의 주소를 대체할 수 없습니다." + +msgid "GPU management" +msgstr "GPU 관리" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Swarm의 일반 리소스는 수동으로 선언하는 스케줄링 리소스이며, 자동 GPU 탐색이" +"나 상태 모니터링을 제공하지 않습니다. 리소스 종류가 ``NVIDIA-GPU`` 이면 Swarm" +"은 할당한 UUID를 ``DOCKER_RESOURCE_NVIDIA-GPU`` 로 전달합니다. NVIDIA runtime" +"도 같은 변수를 읽도록 ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"`` 로 " +"설정해야 합니다. 스케줄링 리소스 예약만으로 GPU 가시성이 제한되지는 않습니다." + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"운영자는 선언한 GPU UUID를 최신으로 유지하고 고장 난 GPU를 처리해야 합니다. " +"구성이 안정적이고 동일한 하드웨어를 사용하는 기여자에게는 수용 가능한 수작업" +"이었지만, 하드웨어가 자주 바뀌면 중요한 절충 요소가 됩니다. 설정된 변수를 선" +"택하는 방식은 `NVIDIA runtime 구현 `_ 을 참고하세요." + +msgid "Persistence and recovery" +msgstr "영속성과 복구" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"현재 Xinference에서 인증을 꺼도 supervisor가 무상태가 되지는 않습니다. 기본 " +"``XINFERENCE_HOME`` 에는 시작 이력(자동 시작 설정 포함), 모니터링 설정, 시스" +"템 설정이 저장됩니다. 이 상태를 영속화하고, 인증을 사용하면 인증 데이터와 키" +"도 영속화하세요. 다운로드 설정의 환경 변수는 이 데이터베이스들을 대체하지 못" +"합니다." + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"호스트 바인드 마운트의 데이터는 해당 호스트에 남습니다. 특정 스토리지 호스트" +"에 배치를 고정하거나 재스케줄링 후에도 접근할 수 있는 적절한 영구 스토리지를 " +"사용하세요. manager가 여러 개인 클러스터에서는 manager 역할 제약만으로 호스" +"트 하나를 지정할 수 없습니다. worker 캐시와 엔진 환경은 별도로 영속화하세요." + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm은 실패한 컨테이너를 교체할 수 있지만, 이것이 Xinference 모델 복구나 무" +"중단 서비스를 보장하지는 않습니다. 선택한 릴리스에서 worker 재연결과 모델 복" +"구를 테스트하세요. manager가 한 대뿐인 Swarm은 그 manager를 사용할 수 없을 " +"때 태스크를 재스케줄링할 수 없지만, 살아 있는 태스크는 계속 실행될 수 있습니" +"다. `Swarm 관리 `_ 를 참고" +"하세요." + +msgid "Further reading" +msgstr "추가 자료" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"오케스트레이션 모델은 `Swarm 개념 `_, 기존 Compose 구성은 :ref:`using_docker_compose`, Xinference " +"3.x 변경 사항은 :doc:`migration_3_0` 을 참고하세요. 완전한 Swarm 구성과 현재 " +"버전의 클러스터 검증은 이 문서의 범위 밖입니다." diff --git a/doc/source/locale/pt_BR/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/pt_BR/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..5ee6eabbca --- /dev/null +++ b/doc/source/locale/pt_BR/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,185 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: pt_BR\n" +"Language: pt_BR\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm: considerações de implantação" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"Este documento é um relato de experiência da comunidade e uma discussão de " +"arquitetura, não um guia de implantação pronto para executar. O colaborador " +"testou Xinference 2.2.0 com Docker 29.2.1, Ubuntu 22.04 e driver NVIDIA 570 " +"em três máquinas, cada uma com uma NVIDIA L20. Xinference 3.x não foi " +"validado nesta configuração Swarm." + +msgid "Why consider Swarm?" +msgstr "Por que considerar Swarm?" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"Para uma equipe pequena que administra cerca de uma dúzia de servidores GPU " +"semelhantes, Swarm pode reduzir o trabalho manual de implantar, reiniciar e " +"atualizar contêineres em várias máquinas. Ele é integrado ao Docker Engine. " +"Esse foi o motivo da escolha do colaborador, não uma regra de que clusters " +"pequenos devam sempre preferir Swarm." + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"Considere a experiência operacional, o gerenciamento de GPUs e os requisitos " +"de disponibilidade, além do tamanho do cluster. Equipes que já operam " +"Kubernetes ou precisam de seu ecossistema de plugins de dispositivos podem " +"preferir :ref:`using_kubernetes`." + +msgid "Two layers of scheduling" +msgstr "Duas camadas de escalonamento" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"Os managers do Swarm alocam tarefas de contêineres nos nós Docker e mantêm o " +"estado desejado dos serviços. Managers também podem executar tarefas. Um " +"supervisor Xinference gerencia a alocação de modelos e as requisições; " +"processos worker Xinference se registram nele e executam modelos. O " +"supervisor não provisiona contêineres worker: a orquestração de contêineres " +"e o serviço de modelos são responsabilidades distintas." + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Um serviço global do Swarm executa uma tarefa por nó elegível, não por GPU. " +"As reservas de GPU e restrições de posicionamento devem corresponder à " +"organização pretendida dos workers. Endereços internos de actors devem " +"identificar processos individuais acessíveis; um endereço de serviço com " +"balanceamento de carga não substitui o endereço de cada worker." + +msgid "GPU management" +msgstr "Gerenciamento de GPUs" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Os recursos genéricos do Swarm são recursos de escalonamento declarados " +"manualmente, não descoberta automática ou monitoramento de integridade de " +"GPUs. Com o tipo ``NVIDIA-GPU``, Swarm exporta os UUIDs alocados por meio de " +"``DOCKER_RESOURCE_NVIDIA-GPU``. O runtime NVIDIA deve ler a mesma variável: " +"``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. Reservas de " +"escalonamento, por si só, não restringem a visibilidade de GPUs." + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"Os operadores devem manter os UUIDs declarados atualizados e lidar com GPUs " +"defeituosas. Esse trabalho manual era aceitável para as máquinas estáveis e " +"homogêneas do colaborador, mas é uma contrapartida importante quando o " +"hardware muda frequentemente. Consulte a `implementação do runtime NVIDIA " +"`_ para saber como a variável " +"configurada é selecionada." + +msgid "Persistence and recovery" +msgstr "Persistência e recuperação" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"No Xinference atual, desativar a autenticação não torna o supervisor sem " +"estado. Seu ``XINFERENCE_HOME`` padrão contém o histórico de inicializações " +"(incluindo a configuração de inicialização automática), a configuração de " +"monitoramento e as configurações do sistema. Persista esse estado; persista " +"também dados de autenticação e chaves quando a autenticação estiver " +"habilitada. Variáveis de ambiente de download não substituem esses bancos de " +"dados." + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"Os dados de um bind mount do host permanecem naquele host. Fixe o " +"posicionamento no host de armazenamento específico ou use armazenamento " +"durável adequado, acessível após o reagendamento. Uma restrição apenas de " +"função manager não seleciona um host único em um cluster com vários " +"managers. Persista separadamente os caches e ambientes de engines dos " +"workers." + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm pode substituir contêineres com falha, mas isso não garante a " +"recuperação dos modelos Xinference nem serviço ininterrupto. Teste a " +"reconexão dos workers e a recuperação de modelos na versão escolhida. Um " +"Swarm com um único manager não pode reagendar tarefas enquanto esse manager " +"estiver indisponível; tarefas sobreviventes podem continuar. Consulte a " +"`administração do Swarm `_." + +msgid "Further reading" +msgstr "Leitura adicional" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"Consulte os `conceitos do Swarm `_ para o modelo de orquestração, :ref:`using_docker_compose` para " +"a configuração Compose existente e :doc:`migration_3_0` para as mudanças do " +"Xinference 3.x. Uma configuração Swarm completa e a validação em cluster da " +"versão atual estão fora do escopo deste relato." diff --git a/doc/source/locale/zh_CN/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/zh_CN/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..3c53bdd5a9 --- /dev/null +++ b/doc/source/locale/zh_CN/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,160 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: zh_CN\n" +"Language: zh_CN\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm：部署考量" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"本文是社区实践经验与架构讨论，不是可直接执行的部署指南。贡献者使用 " +"Xinference 2.2.0、Docker 29.2.1、Ubuntu 22.04 和 NVIDIA 570 驱动，在三台各配" +"一张 NVIDIA L20 的机器上进行了测试。此 Swarm 方案尚未验证 Xinference 3.x。" + +msgid "Why consider Swarm?" +msgstr "为什么考虑 Swarm？" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"对于管理十余台配置相近的 GPU 服务器的小团队，Swarm 可以减少跨机器部署、重启和" +"更新容器的手工操作。它内置于 Docker Engine。这是贡献者选择它的原因，并不意味" +"着小集群总应优先使用 Swarm。" + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"除集群规模外，还应考虑现有运维经验、GPU 管理和可用性要求。已运维 Kubernetes " +"或需要其设备插件生态的团队，可以优先考虑 :ref:`using_kubernetes`。" + +msgid "Two layers of scheduling" +msgstr "两层调度" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"Swarm manager 在 Docker 节点上调度容器任务，维持服务的期望状态。manager 也可" +"以运行任务。Xinference supervisor 管理模型放置和请求；Xinference worker 进程" +"向它注册并运行模型。supervisor 不负责创建 worker 容器：容器编排与模型服务是不" +"同的职责。" + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Swarm 的 global 服务在每个符合条件的节点上运行一个任务，而不是每张 GPU 一个任" +"务。GPU 资源预留和放置约束必须符合预期的 worker 布局。内部 actor 地址必须标识" +"可访问的具体进程；负载均衡的服务地址不能代替每个 worker 自身的地址。" + +msgid "GPU management" +msgstr "GPU 管理" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Swarm 通用资源是手动声明的调度资源，不提供自动 GPU 发现或健康监测。资源类型" +"为 ``NVIDIA-GPU`` 时，Swarm 通过 ``DOCKER_RESOURCE_NVIDIA-GPU`` 导出分配的 " +"UUID。NVIDIA runtime 必须配置为读取同一变量：``swarm-resource = " +"\"DOCKER_RESOURCE_NVIDIA-GPU\"``。仅有调度资源预留并不能限制 GPU 可见性。" + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"运维人员必须更新声明的 GPU UUID，并处理故障 GPU。对于贡献者配置稳定、硬件一致" +"的机器，这些手工工作可以接受；但硬件经常变化时，这是重要的取舍。配置变量的选" +"择方式见 `NVIDIA runtime 实现 `_。" + +msgid "Persistence and recovery" +msgstr "持久化与恢复" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"在当前 Xinference 中，关闭认证并不会让 supervisor 变成无状态服务。默认的 " +"``XINFERENCE_HOME`` 包含启动历史（包括自动启动配置）、监控配置和系统设置。这" +"些状态需要持久化；启用认证时，还需持久化认证数据和密钥。下载设置的环境变量不" +"能替代这些数据库。" + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"主机绑定挂载的数据留在该主机上。应将服务固定在具体的存储主机，或使用重新调度" +"后仍可访问的合适持久存储。在多 manager 集群中，仅限制 manager 角色不能指定唯" +"一主机。worker 的缓存和引擎环境需要单独持久化。" + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm 可以替换故障容器，但这不等于 Xinference 的模型恢复或服务不中断。应针对" +"所选版本测试 worker 重连和模型恢复。单 manager Swarm 在该 manager 不可用时无" +"法重新调度任务；其他仍存活的任务可以继续运行。参见 `Swarm 管理 `_。" + +msgid "Further reading" +msgstr "延伸阅读" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"编排模型见 `Swarm 概念 `_，现有 Compose 方案见 :ref:`using_docker_compose`，Xinference 3.x 的变化" +"见 :doc:`migration_3_0`。完整的 Swarm 配置和当前版本的集群验证不在本文范围" +"内。" diff --git a/doc/source/locale/zh_TW/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/zh_TW/LC_MESSAGES/getting_started/using_docker_swarm.po new file mode 100644 index 0000000000..ab5a5013b9 --- /dev/null +++ b/doc/source/locale/zh_TW/LC_MESSAGES/getting_started/using_docker_swarm.po @@ -0,0 +1,160 @@ +# Copyright (C) 2026, XINFERENCE HOLDINGS PTE. LTD. +# Distributed under the same license as Xinference. +msgid "" +msgstr "" +"Project-Id-Version: Xinference\n" +"Report-Msgid-Bugs-To: https://github.com/xorbitsai/inference/issues\n" +"POT-Creation-Date: 2026-10-04 00:00+0800\n" +"PO-Revision-Date: 2026-10-04 00:00+0800\n" +"Last-Translator: Xinference contributors\n" +"Language-Team: zh_TW\n" +"Language: zh_TW\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" + +msgid "Docker Swarm: Deployment Considerations" +msgstr "Docker Swarm：部署考量" + +msgid "" +"This is a community experience report and architecture discussion, not a " +"ready-to-run deployment guide. The contributor tested Xinference 2.2.0 with " +"Docker 29.2.1, Ubuntu 22.04 and NVIDIA driver 570 on three machines, each " +"with one NVIDIA L20. Xinference 3.x has not been validated in this Swarm " +"setup." +msgstr "" +"本文是社群實務經驗與架構討論，不是可直接執行的部署指南。貢獻者使用 " +"Xinference 2.2.0、Docker 29.2.1、Ubuntu 22.04 和 NVIDIA 570 驅動程式，在三台" +"各配一張 NVIDIA L20 的機器上進行測試。此 Swarm 方案尚未驗證 Xinference 3.x。" + +msgid "Why consider Swarm?" +msgstr "為什麼考慮 Swarm？" + +msgid "" +"For a small team managing a dozen or so similar GPU servers, Swarm can " +"reduce the manual work of deploying, restarting and updating containers " +"across machines. It is built into Docker Engine. This was the contributor's " +"reason for choosing it, not a rule that small clusters should always prefer " +"Swarm." +msgstr "" +"對於管理十餘台組態相近的 GPU 伺服器的小團隊，Swarm 可以減少跨機器部署、重新啟" +"動和更新容器的手動操作。它內建於 Docker Engine。這是貢獻者選擇它的原因，並不" +"代表小叢集總應優先使用 Swarm。" + +msgid "" +"Consider existing operational expertise, GPU management and availability " +"requirements as well as cluster size. Teams already operating Kubernetes or " +"needing its device-plugin ecosystem may prefer :ref:`using_kubernetes`." +msgstr "" +"除了叢集規模，還應考慮現有維運經驗、GPU 管理及可用性需求。已維運 Kubernetes " +"或需要其裝置外掛生態系的團隊，可以優先考慮 :ref:`using_kubernetes`。" + +msgid "Two layers of scheduling" +msgstr "兩層排程" + +msgid "" +"Swarm managers schedule container tasks on Docker nodes and maintain the " +"services' desired state. Managers can also run tasks. A Xinference " +"supervisor manages model placement and requests; Xinference worker processes " +"register with it and run models. The supervisor does not provision worker " +"containers: container orchestration and model serving are separate " +"responsibilities." +msgstr "" +"Swarm manager 在 Docker 節點上排程容器任務，維持服務的期望狀態。manager 也可" +"以執行任務。Xinference supervisor 管理模型配置位置與請求；Xinference worker " +"程序向它註冊並執行模型。supervisor 不負責建立 worker 容器：容器編排與模型服務" +"是不同職責。" + +msgid "" +"A global Swarm service runs one task per eligible node, not one per GPU. GPU " +"reservations and placement constraints must match the intended worker " +"layout. Internal actor addresses must identify reachable individual " +"processes; a load-balanced service address is not a substitute for each " +"worker's address." +msgstr "" +"Swarm 的 global 服務在每個符合條件的節點上執行一個任務，而不是每張 GPU 一個任" +"務。GPU 資源預留與放置約束必須符合預期的 worker 配置。內部 actor 位址必須識別" +"可連線的特定程序；負載平衡的服務位址不能取代每個 worker 自身的位址。" + +msgid "GPU management" +msgstr "GPU 管理" + +msgid "" +"Swarm generic resources are manually advertised scheduling resources, not " +"automatic GPU discovery or health monitoring. With the resource kind " +"``NVIDIA-GPU``, Swarm exports allocated UUIDs through " +"``DOCKER_RESOURCE_NVIDIA-GPU``. NVIDIA runtime configuration must read that " +"same variable: ``swarm-resource = \"DOCKER_RESOURCE_NVIDIA-GPU\"``. " +"Scheduling reservations alone do not enforce GPU visibility." +msgstr "" +"Swarm 通用資源是手動宣告的排程資源，不提供自動 GPU 探索或健康監測。資源類型" +"為 ``NVIDIA-GPU`` 時，Swarm 透過 ``DOCKER_RESOURCE_NVIDIA-GPU`` 匯出分配的 " +"UUID。NVIDIA runtime 必須設定為讀取同一變數：``swarm-resource = " +"\"DOCKER_RESOURCE_NVIDIA-GPU\"``。僅有排程資源預留並不能限制 GPU 可見性。" + +msgid "" +"Operators must keep advertised GPU UUIDs current and handle failed GPUs. " +"This manual work was acceptable for the contributor's stable, homogeneous " +"machines, but it is an important trade-off when hardware changes frequently. " +"See the `NVIDIA runtime implementation `_ for how the configured variable is selected." +msgstr "" +"維運人員必須更新宣告的 GPU UUID，並處理故障 GPU。對於貢獻者組態穩定、硬體一致" +"的機器，這些手動工作可以接受；但硬體經常變動時，這是重要的取捨。設定變數的選" +"擇方式見 `NVIDIA runtime 實作 `_。" + +msgid "Persistence and recovery" +msgstr "持久化與復原" + +msgid "" +"In current Xinference, disabling authentication does not make the supervisor " +"stateless. Its default ``XINFERENCE_HOME`` contains launch history " +"(including autostart configuration), monitoring configuration and system " +"settings. Persist this state; persist authentication data and keys too when " +"authentication is enabled. Environment variables for download settings do " +"not replace these databases." +msgstr "" +"在目前的 Xinference 中，關閉驗證並不會讓 supervisor 成為無狀態服務。預設的 " +"``XINFERENCE_HOME`` 包含啟動歷史（包括自動啟動設定）、監控設定與系統設定。這" +"些狀態需要持久化；啟用驗證時，還需持久化驗證資料與金鑰。下載設定的環境變數不" +"能取代這些資料庫。" + +msgid "" +"A host bind mount stays on that host. Use placement on the specific storage " +"host, or suitable durable storage available after rescheduling. A manager-" +"role constraint alone does not select a unique host in a multi-manager " +"cluster. Persist worker caches and engine environments separately." +msgstr "" +"主機繫結掛載的資料留在該主機上。應將服務固定於特定儲存主機，或使用重新排程後" +"仍可存取的合適持久儲存。在多 manager 叢集中，僅限制 manager 角色無法指定唯一" +"主機。worker 的快取與引擎環境需另外持久化。" + +msgid "" +"Swarm can replace failed containers, but that does not establish Xinference " +"model recovery or uninterrupted service. Test worker reconnection and model " +"recovery for the chosen release. A single-manager Swarm cannot reschedule " +"tasks while that manager is unavailable; surviving tasks can continue " +"running. See `Swarm administration `_." +msgstr "" +"Swarm 可以替換故障容器，但這不等於 Xinference 的模型復原或服務不中斷。應針對" +"所選版本測試 worker 重新連線與模型復原。單 manager Swarm 在該 manager 無法使" +"用時不能重新排程任務；其他仍存活的任務可以繼續執行。參見 `Swarm 管理 " +"`_。" + +msgid "Further reading" +msgstr "延伸閱讀" + +msgid "" +"See `Swarm concepts `_ " +"for the orchestration model, :ref:`using_docker_compose` for the existing " +"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"complete Swarm configuration and current-version cluster validation are " +"outside the scope of this report." +msgstr "" +"編排模型見 `Swarm 概念 `_，現有 Compose 方案見 :ref:`using_docker_compose`，Xinference 3.x 的變更" +"見 :doc:`migration_3_0`。完整的 Swarm 設定與目前版本的叢集驗證不在本文範圍" +"內。" From 3c23aaa7d6123d581cb4206bd06cde16fda08ace Mon Sep 17 00:00:00 2001 From: qinxuye Date: Sun, 4 Oct 2026 18:53:31 +0800 Subject: [PATCH 3/3] docs: move Swarm considerations to user guide deployment --- doc/source/getting_started/index.rst | 1 - doc/source/locale/de/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/locale/es/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/locale/fr/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/locale/it/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/locale/ja/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/locale/ko/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/locale/pt_BR/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/locale/zh_TW/LC_MESSAGES/user_guide/index.po | 4 ++++ .../{getting_started => user_guide}/using_docker_swarm.po | 4 ++-- doc/source/user_guide/index.rst | 6 ++++++ .../{getting_started => user_guide}/using_docker_swarm.rst | 2 +- 21 files changed, 61 insertions(+), 20 deletions(-) rename doc/source/locale/de/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (98%) rename doc/source/locale/es/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (98%) rename doc/source/locale/fr/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (98%) rename doc/source/locale/it/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (98%) rename doc/source/locale/ja/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (98%) rename doc/source/locale/ko/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (97%) rename doc/source/locale/pt_BR/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (98%) rename doc/source/locale/zh_CN/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (97%) rename doc/source/locale/zh_TW/LC_MESSAGES/{getting_started => user_guide}/using_docker_swarm.po (97%) rename doc/source/{getting_started => user_guide}/using_docker_swarm.rst (94%) diff --git a/doc/source/getting_started/index.rst b/doc/source/getting_started/index.rst index 53abccccfb..24d3bf8823 100644 --- a/doc/source/getting_started/index.rst +++ b/doc/source/getting_started/index.rst @@ -13,7 +13,6 @@ Getting Started logging using_docker_image using_docker_compose - using_docker_swarm using_kubernetes troubleshooting environments diff --git a/doc/source/locale/de/LC_MESSAGES/user_guide/index.po b/doc/source/locale/de/LC_MESSAGES/user_guide/index.po index 516b7f526f..fd4572a17a 100644 --- a/doc/source/locale/de/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/de/LC_MESSAGES/user_guide/index.po @@ -26,6 +26,10 @@ msgstr "Benutzerhandbuch" msgid "Model usage" msgstr "Modellnutzung" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "Bereitstellung" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "Betrieb und Sicherheit" diff --git a/doc/source/locale/de/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/de/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 98% rename from doc/source/locale/de/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/de/LC_MESSAGES/user_guide/using_docker_swarm.po index 92f32edaa7..65d0e3d803 100644 --- a/doc/source/locale/de/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/de/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -174,12 +174,12 @@ msgstr "Weiterführende Informationen" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "Siehe `Swarm-Konzepte `_ " "zum Orchestrierungsmodell, :ref:`using_docker_compose` zur vorhandenen " -"Compose-Konfiguration und :doc:`migration_3_0` zu Änderungen in Xinference " +"Compose-Konfiguration und :doc:`/getting_started/migration_3_0` zu Änderungen in Xinference " "3.x. Eine vollständige Swarm-Konfiguration und Cluster-Validierung der " "aktuellen Version liegen außerhalb dieses Berichts." diff --git a/doc/source/locale/es/LC_MESSAGES/user_guide/index.po b/doc/source/locale/es/LC_MESSAGES/user_guide/index.po index cf718d7af9..9d58472b36 100644 --- a/doc/source/locale/es/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/es/LC_MESSAGES/user_guide/index.po @@ -27,6 +27,10 @@ msgstr "GUÍA DEL USUARIO" msgid "Model usage" msgstr "Uso de modelos" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "Despliegue" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "Operaciones y seguridad" diff --git a/doc/source/locale/es/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/es/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 98% rename from doc/source/locale/es/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/es/LC_MESSAGES/user_guide/using_docker_swarm.po index 8d348f41e9..6b5eafcad8 100644 --- a/doc/source/locale/es/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/es/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -175,12 +175,12 @@ msgstr "Lecturas adicionales" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "Consulte los `conceptos de Swarm `_ para el modelo de orquestación, :ref:`using_docker_compose` " -"para la configuración Compose existente y :doc:`migration_3_0` para los " +"para la configuración Compose existente y :doc:`/getting_started/migration_3_0` para los " "cambios de Xinference 3.x. Una configuración Swarm completa y la validación " "en clúster de la versión actual quedan fuera del alcance de este informe." diff --git a/doc/source/locale/fr/LC_MESSAGES/user_guide/index.po b/doc/source/locale/fr/LC_MESSAGES/user_guide/index.po index 389c7eb338..7bfc9753ab 100644 --- a/doc/source/locale/fr/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/fr/LC_MESSAGES/user_guide/index.po @@ -27,6 +27,10 @@ msgstr "Guide de l'utilisateur" msgid "Model usage" msgstr "Utilisation des modèles" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "Déploiement" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "Exploitation et sécurité" diff --git a/doc/source/locale/fr/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/fr/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 98% rename from doc/source/locale/fr/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/fr/LC_MESSAGES/user_guide/using_docker_swarm.po index 02ce8b024b..705222bb45 100644 --- a/doc/source/locale/fr/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/fr/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -175,13 +175,13 @@ msgstr "Pour aller plus loin" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "Consultez les `concepts de Swarm `_ pour le modèle d'orchestration, :ref:`using_docker_compose` " -"pour la configuration Compose existante et :doc:`migration_3_0` pour les " +"pour la configuration Compose existante et :doc:`/getting_started/migration_3_0` pour les " "changements de Xinference 3.x. Une configuration Swarm complète et la " "validation en cluster de la version actuelle sont hors du périmètre de ce " "rapport." diff --git a/doc/source/locale/it/LC_MESSAGES/user_guide/index.po b/doc/source/locale/it/LC_MESSAGES/user_guide/index.po index 029d926e20..c9c0123899 100644 --- a/doc/source/locale/it/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/it/LC_MESSAGES/user_guide/index.po @@ -26,6 +26,10 @@ msgstr "Guida per l'utente" msgid "Model usage" msgstr "Utilizzo dei modelli" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "Deployment" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "Gestione e sicurezza" diff --git a/doc/source/locale/it/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/it/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 98% rename from doc/source/locale/it/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/it/LC_MESSAGES/user_guide/using_docker_swarm.po index 5ebeaf6730..b5a9aaf0b7 100644 --- a/doc/source/locale/it/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/it/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -175,13 +175,13 @@ msgstr "Approfondimenti" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "Consultate i `concetti di Swarm `_ per il modello di orchestrazione, :ref:`using_docker_compose` " -"per la configurazione Compose esistente e :doc:`migration_3_0` per le " +"per la configurazione Compose esistente e :doc:`/getting_started/migration_3_0` per le " "modifiche di Xinference 3.x. Una configurazione Swarm completa e la " "validazione in cluster della versione attuale non rientrano nell'ambito di " "questo resoconto." diff --git a/doc/source/locale/ja/LC_MESSAGES/user_guide/index.po b/doc/source/locale/ja/LC_MESSAGES/user_guide/index.po index ed9867c48f..056c438d7a 100644 --- a/doc/source/locale/ja/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/ja/LC_MESSAGES/user_guide/index.po @@ -26,6 +26,10 @@ msgstr "ユーザーガイド" msgid "Model usage" msgstr "モデルの利用" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "デプロイ" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "運用とセキュリティ" diff --git a/doc/source/locale/ja/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/ja/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 98% rename from doc/source/locale/ja/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/ja/LC_MESSAGES/user_guide/using_docker_swarm.po index 2ebaa0c238..477bc10a89 100644 --- a/doc/source/locale/ja/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/ja/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -162,11 +162,11 @@ msgstr "関連資料" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "オーケストレーションの仕組みは `Swarm の概念 `_、既存の Compose 構成は :ref:`using_docker_compose`、" -"Xinference 3.x の変更は :doc:`migration_3_0` を参照してください。完全な " +"Xinference 3.x の変更は :doc:`/getting_started/migration_3_0` を参照してください。完全な " "Swarm 構成と現行バージョンのクラスター検証は本稿の対象外です。" diff --git a/doc/source/locale/ko/LC_MESSAGES/user_guide/index.po b/doc/source/locale/ko/LC_MESSAGES/user_guide/index.po index 249211e533..8b25fdd485 100644 --- a/doc/source/locale/ko/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/ko/LC_MESSAGES/user_guide/index.po @@ -27,6 +27,10 @@ msgstr "사용자 가이드" msgid "Model usage" msgstr "모델 사용" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "배포" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "운영 및 보안" diff --git a/doc/source/locale/ko/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/ko/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 97% rename from doc/source/locale/ko/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/ko/LC_MESSAGES/user_guide/using_docker_swarm.po index d058c8beb5..a1c7167bd6 100644 --- a/doc/source/locale/ko/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/ko/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -162,11 +162,11 @@ msgstr "추가 자료" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "오케스트레이션 모델은 `Swarm 개념 `_, 기존 Compose 구성은 :ref:`using_docker_compose`, Xinference " -"3.x 변경 사항은 :doc:`migration_3_0` 을 참고하세요. 완전한 Swarm 구성과 현재 " +"3.x 변경 사항은 :doc:`/getting_started/migration_3_0` 을 참고하세요. 완전한 Swarm 구성과 현재 " "버전의 클러스터 검증은 이 문서의 범위 밖입니다." diff --git a/doc/source/locale/pt_BR/LC_MESSAGES/user_guide/index.po b/doc/source/locale/pt_BR/LC_MESSAGES/user_guide/index.po index 39606c59ec..2a0759833c 100644 --- a/doc/source/locale/pt_BR/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/pt_BR/LC_MESSAGES/user_guide/index.po @@ -27,6 +27,10 @@ msgstr "Guia do Usuário" msgid "Model usage" msgstr "Uso de modelos" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "Implantação" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "Operações e segurança" diff --git a/doc/source/locale/pt_BR/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/pt_BR/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 98% rename from doc/source/locale/pt_BR/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/pt_BR/LC_MESSAGES/user_guide/using_docker_swarm.po index 5ee6eabbca..7565a80e12 100644 --- a/doc/source/locale/pt_BR/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/pt_BR/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -174,12 +174,12 @@ msgstr "Leitura adicional" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "Consulte os `conceitos do Swarm `_ para o modelo de orquestração, :ref:`using_docker_compose` para " -"a configuração Compose existente e :doc:`migration_3_0` para as mudanças do " +"a configuração Compose existente e :doc:`/getting_started/migration_3_0` para as mudanças do " "Xinference 3.x. Uma configuração Swarm completa e a validação em cluster da " "versão atual estão fora do escopo deste relato." diff --git a/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po index 9368a51db1..d5dd809974 100644 --- a/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/index.po @@ -28,6 +28,10 @@ msgstr "用户指南" msgid "Model usage" msgstr "模型使用" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "部署" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "运维与安全" diff --git a/doc/source/locale/zh_CN/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 97% rename from doc/source/locale/zh_CN/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/zh_CN/LC_MESSAGES/user_guide/using_docker_swarm.po index 3c53bdd5a9..22aba8e39e 100644 --- a/doc/source/locale/zh_CN/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/zh_CN/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -150,11 +150,11 @@ msgstr "延伸阅读" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "编排模型见 `Swarm 概念 `_，现有 Compose 方案见 :ref:`using_docker_compose`，Xinference 3.x 的变化" -"见 :doc:`migration_3_0`。完整的 Swarm 配置和当前版本的集群验证不在本文范围" +"见 :doc:`/getting_started/migration_3_0`。完整的 Swarm 配置和当前版本的集群验证不在本文范围" "内。" diff --git a/doc/source/locale/zh_TW/LC_MESSAGES/user_guide/index.po b/doc/source/locale/zh_TW/LC_MESSAGES/user_guide/index.po index 43929c2438..d27647ef61 100644 --- a/doc/source/locale/zh_TW/LC_MESSAGES/user_guide/index.po +++ b/doc/source/locale/zh_TW/LC_MESSAGES/user_guide/index.po @@ -27,6 +27,10 @@ msgstr "使用者指南" msgid "Model usage" msgstr "模型使用" +#: ../../source/user_guide/index.rst +msgid "Deployment" +msgstr "部署" + #: ../../source/user_guide/index.rst msgid "Operations and security" msgstr "維運與安全" diff --git a/doc/source/locale/zh_TW/LC_MESSAGES/getting_started/using_docker_swarm.po b/doc/source/locale/zh_TW/LC_MESSAGES/user_guide/using_docker_swarm.po similarity index 97% rename from doc/source/locale/zh_TW/LC_MESSAGES/getting_started/using_docker_swarm.po rename to doc/source/locale/zh_TW/LC_MESSAGES/user_guide/using_docker_swarm.po index ab5a5013b9..da02d87e44 100644 --- a/doc/source/locale/zh_TW/LC_MESSAGES/getting_started/using_docker_swarm.po +++ b/doc/source/locale/zh_TW/LC_MESSAGES/user_guide/using_docker_swarm.po @@ -150,11 +150,11 @@ msgstr "延伸閱讀" msgid "" "See `Swarm concepts `_ " "for the orchestration model, :ref:`using_docker_compose` for the existing " -"Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A " +"Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A " "complete Swarm configuration and current-version cluster validation are " "outside the scope of this report." msgstr "" "編排模型見 `Swarm 概念 `_，現有 Compose 方案見 :ref:`using_docker_compose`，Xinference 3.x 的變更" -"見 :doc:`migration_3_0`。完整的 Swarm 設定與目前版本的叢集驗證不在本文範圍" +"見 :doc:`/getting_started/migration_3_0`。完整的 Swarm 設定與目前版本的叢集驗證不在本文範圍" "內。" diff --git a/doc/source/user_guide/index.rst b/doc/source/user_guide/index.rst index 369ffeb761..8083a23268 100644 --- a/doc/source/user_guide/index.rst +++ b/doc/source/user_guide/index.rst @@ -13,6 +13,12 @@ User Guide client_api SD WebUI API compatibility +.. toctree:: + :maxdepth: 2 + :caption: Deployment + + using_docker_swarm + .. toctree:: :maxdepth: 2 :caption: Operations and security diff --git a/doc/source/getting_started/using_docker_swarm.rst b/doc/source/user_guide/using_docker_swarm.rst similarity index 94% rename from doc/source/getting_started/using_docker_swarm.rst rename to doc/source/user_guide/using_docker_swarm.rst index fc3fbb9fde..bd9893c85c 100644 --- a/doc/source/getting_started/using_docker_swarm.rst +++ b/doc/source/user_guide/using_docker_swarm.rst @@ -38,4 +38,4 @@ Swarm can replace failed containers, but that does not establish Xinference mode Further reading --------------- -See `Swarm concepts `_ for the orchestration model, :ref:`using_docker_compose` for the existing Compose setup, and :doc:`migration_3_0` for Xinference 3.x changes. A complete Swarm configuration and current-version cluster validation are outside the scope of this report. +See `Swarm concepts `_ for the orchestration model, :ref:`using_docker_compose` for the existing Compose setup, and :doc:`/getting_started/migration_3_0` for Xinference 3.x changes. A complete Swarm configuration and current-version cluster validation are outside the scope of this report.