feat: add cross-platform installation and local service management

[OliverBryant]

Summary

The convenience installer now installs and starts a single-machine Xinference server on Linux, macOS, and Windows. It prepares uv/Python automatically and uses a persistent uv tool install environment, with hardware-aware PyTorch selection on Linux/Windows and optional version/backend/extras overrides.

Adds xinference service install/start/stop/restart/status/logs/uninstall for existing pip, Conda, and uv installations. Linux uses systemd, macOS uses launchd, and x86-64 Windows uses a pinned, SHA-256-verified WinSW wrapper. Startup waits for /status; shutdown waits for the captured service process tree to exit. Repeated installation reuses matching settings, unmanaged services are preserved, and uninstall keeps model data and logs.

Installer bootstrap, package probes, and service commands use isolated Python imports. Windows service control files and working directory live in Program Files, while model data remains separate. The service identity remains LocalSystem with administrator-controlled runtime/model inputs. Windows registration warns that the one-command installer uses the installing user's tool store and does not restrict runtime/data ACLs. Windows foreground startup waits for the child and returns its exit code; piped PowerShell execution keeps caller variables local, including Windows PowerShell 5.1. The tool-store override is XINFERENCE_TOOL_DIR, leaving Docker's XINFERENCE_HOME_DIR unchanged.

Rerunning the installer updates to the latest stable release, including an installation that was pinned on a previous run. Set XINFERENCE_VERSION each time to select a target release or downgrade. A shared standalone driver prepares and checks the candidate before stopping the service, backs up the complete previous environment and command shims, and restores the old environment and previous service state if installation or native service startup fails. Interrupted transactions retain recovery files and recover before the next attempt, including after a service was uninstalled. A service installer API check rejects incompatible candidates before stopping a service. Concurrent installers are rejected.

Existing managed services are detected automatically. Their account, host, port, data directory, and registration are preserved, as are Python, extras, and the PyTorch backend unless overridden. Repeats with the same version, Python, extras and backend keep the environment and do not restart a running service. A separate foreground process blocks replacement of the environment. Failed startup removes a newly created service registration, including when the package is unchanged. Registration ownership is saved atomically before the native definition so an interrupted definition write can be retried. Service restarts preserve cached data but require previously running models to be launched again; foreground startup has no automatic readiness rollback.

Usage

Linux/macOS:

curl -fsSL https://raw.githubusercontent.com/xorbitsai/inference/main/scripts/install.sh | sh

Windows PowerShell:

irm https://raw.githubusercontent.com/xorbitsai/inference/main/scripts/install.ps1 | iex

The first installation starts in the foreground by default; later runs detect an existing service through XINFERENCE_SERVICE=auto. Set XINFERENCE_START=0 for installation only (an upgraded service stays stopped). Set XINFERENCE_SERVICE=user on Linux/macOS or XINFERENCE_SERVICE=system on any supported platform to register and start a service. System mode installs as your account and uses sudo on Linux/macOS; Windows requires an Administrator terminal. Existing installations can register Linux/macOS system services through xinference service --system install --user USER --start with an elevated terminal and the installed command's absolute path.

Initial installation uses the base framework; model environments install supported engine dependencies on demand. Existing platform restrictions remain in place. Services require a release containing the new CLI; CI installs the candidate wheel rather than an older PyPI release. README and installation documentation include corresponding updates in all nine existing locales.

Validation

• 68 focused installer/service/CLI tests passed, including pipeline/import isolation, normalized settings, first-install service cleanup, Windows foreground exit propagation, environment/shim rollback, interrupted recovery, concurrency, and service/foreground process ownership. The existing complete CLI regression module also passed in an earlier combined local run.
• An isolated offline smoke test with real uv installations passed initial installation, latest-version upgrade, same-version repeat, downgrade, and upgrade from a previous pin.
• Pre-commit, shellcheck, Python script lint, and diff checks passed.
• All modified catalogs passed msgfmt --check --check-format, compiled via doc/build_i18n.py --all, and the installation page rendered in English and every maintained locale.
• Installer CI is running at 2a658b2c7022a58e6249678d154cb0ced041f4b7. It builds and installs the candidate wheel and verifies native system-service startup/restart/stop/uninstall, process cleanup, auto-detection, same-version PID preservation, latest-version upgrade, deliberate startup failure and rollback, downgrade, data preservation, and paths containing spaces. It also rejects a deliberately incompatible service API without changing the running PID. Windows uses piped Windows PowerShell 5.1 invocation with caller-scope assertions and a planted module in model data. These CPU checks use NO_WEB_UI=1; no real GPU/Metal model inference is exercised.

[gemini-code-assist]

Code Review

This pull request introduces a new one-line installation script (scripts/install.sh) for Linux and macOS that installs Xinference as an isolated tool using uv, and updates the README and documentation accordingly. The review feedback correctly identifies that the command uv tool update-shell referenced in the script does not exist, and provides a robust code suggestion to simplify the PATH warning logic.

[qinxuye]

Thanks for adding this. I think the scope needs to be clarified before this becomes the primary Quick Start installer. The base xinference package already depends on torch, so even the default installation is hardware-sensitive: on Linux we need to account for CPU, NVIDIA, AMD/ROCm, Intel GPU, and the driver-supported PyTorch backend. Please use hardware-aware backend selection such as uv --torch-backend auto, allow an explicit override, and keep hardware-specific backends such as vLLM on a clearly supported path. macOS also needs to distinguish Apple Silicon from Intel. Since uv tool creates an isolated environment, packages from an active Conda environment or virtualenv are not reused; the script should at least detect an existing xinference command and warn about PATH/environment conflicts. If the intended scope is only installing the CLI/server framework, the README should state clearly that common model backends are not installed and this should not replace the existing functional Quick Start. Otherwise, please provide explicit CPU/CUDA/ROCm/MLX modes, select a tested Python version, perform preflight checks, and add CI coverage that actually exercises the installer rather than only repository lint/tests.

[OliverBryant]

Thanks @qinxuye, this was the right call — the base package pulls in torch, so a naive uv tool install is indeed hardware-blind. I've reworked the installer to address each point:

Hardware-aware PyTorch. The script now creates a dedicated uv-managed virtualenv and installs via uv pip install --torch-backend "$XINFERENCE_BACKEND" .... --torch-backend is only available in the uv pip interface (not uv tool install), which is why I moved off the tool path. XINFERENCE_BACKEND defaults to auto (uv detects CUDA/ROCm/Intel driver and picks the matching index) and accepts explicit cpu / cuXXX / rocm / xpu overrides.

macOS Apple Silicon vs Intel. uname -m is checked; on x86_64 macOS the script warns that recent PyTorch no longer ships Intel-mac wheels. --torch-backend is skipped on macOS (single universal wheel).

Isolated env / conflict detection. Preflight now warns when an existing xinference is on PATH (it will be shadowed via ~/.local/bin), and when run inside an active virtualenv/conda env it warns that uv installs into a separate env and packages are not reused, pointing to pip install "xinference[all]" for in-place installs.

Scope is explicit. This installs only the base framework — no model backends. Both the README and the installation docs now state this clearly and position it as a convenience bootstrap, not a replacement for the existing pip install "xinference[all]" Quick Start, which is kept as the functional path. Extras remain opt-in via XINFERENCE_EXTRAS.

Pinned Python + preflight + CI. The venv pins Python (default 3.12, override via XINFERENCE_PYTHON). A new install-script.yaml workflow shellchecks the script and runs it end-to-end in CPU mode on Linux and macOS, asserting xinference-local --help and xinference --version work — so the installer itself is exercised, not just repo lint/tests.

I verified the flow end-to-end locally on macOS arm64 (installs base xinference, links the commands, xinference-local --help returns 0). vLLM/ROCm/XPU paths are documented but not exercised in CI (no such hardware on hosted runners). Let me know if you'd prefer a different default backend or Python version.

[rogercloud]

Approach: acceptable with reservations. Two local privilege-escalation paths block merge; the rest are robustness, UX and follow-ups.

Major

• scripts/install.sh:36: under curl | sh, the driver runs from shared /tmp without -I, so a planted /tmp/json.py gets code execution (see inline).
• xinference/deploy/service.py:331: the Windows system service runs as LocalSystem with python -c and cwd set to a data dir that ordinary users can write to, so a planted module gets SYSTEM code execution (see inline).

Minor

• scripts/install.sh:30: under curl | sh a ./manage_install.py in the cwd is executed, and the second trap ... 0 at :33 drops the $uv_script cleanup (see inline).
• scripts/manage_install.py:66/:390: python -c probes put the cwd first on sys.path, so the candidate check imports ./xinference, and the sudo path imports from the user's cwd as root (see inline).
• scripts/manage_install.py:501: on Windows os.execve spawns and exits, so the documented foreground "Ctrl+C to stop" does not hold (see inline).
• scripts/manage_install.py:98: XINFERENCE_HOME_DIR already means the host data dir in xinference/deploy/docker/docker-compose.yml:44 (see inline).
• scripts/manage_install.py:185: comparing raw strings against the normalized stored config makes every upgrade abort (see inline).
• scripts/manage_install.py:413: copytree backs up a multi-GB env while the staged env also exists, so peak disk use is about 3x (see inline).
• scripts/manage_install.py:24-28: uv output is captured and only printed after it exits, so a multi-GB torch download shows no progress. Stream the install_tool output.
• scripts/manage_install.py:409/:469: a first install has no rollback, so a failed service start leaves the service registered and enabled. Uninstall when the service was created in this run, or print a hint. Scope the restore wording at installation.rst:67 to upgrades.
• scripts/manage_install.py:372: the foreground-busy check runs before the unchanged comparison, so a no-op rerun fails. It is also skipped when a service exists, so a separate foreground process on the same env is missed. Exclude only the service PIDs and check only on the replace path.
• xinference/deploy/service.py:440: on timeout, start() silently stops a healthy but slow service, and the CLI --timeout default is 60 while the installer uses 120 (see inline).
• scripts/install.ps1:3: under irm | iex, $ErrorActionPreference='Stop' and the script variables leak into the user's session. Wrap the script in & { ... }.
• xinference/deploy/service.py:158: per-user macOS mode uses gui/<uid>, which fails over SSH on a headless Mac, and launchd console.log is never rotated. Document this or add a fallback.
• Tests: the test_installer.py fixture stubs run/install_tool/active, so the install_tool argv (Darwin omits --torch-backend) and the per-platform active() parsing (manage_install.py:223-247) are untested. In test_service.py:428-461, 4 modes are parametrized but install.sh never branches on mode.
• CI: install-script.yaml uses actions/checkout@v4 (other workflows use v7) and sets no permissions: contents: read. scripts/check_install.py:317 finally: run("uninstall") can mask the original error.
• [prior] I saw your reply that the preflight warns about an existing xinference on PATH. I re-checked scripts/install.sh:13-15, scripts/install.ps1 and scripts/manage_install.py. Only an active venv/conda prints a note, and nothing detects or warns about a conflicting xinference.
• [prior] There is no Intel-Mac (x86_64 Darwin) preflight. scripts/install.sh:9-12 checks only the OS, so the failure only shows up when uv cannot resolve torch, even though installation.rst:53 says Apple Silicon is required.
• [prior] The README installer section (~L180-190) still does not say that it installs only the base framework and does not replace the xinference[all] Quick Start, as promised in the reply.

Blocking: yes — recommended event: REQUEST_CHANGES

• scripts/install.sh:36, major: another local user can run code as the installing user, or as root in system mode. [new]
• xinference/deploy/service.py:331, major: an unprivileged local user can run code as SYSTEM on Windows hosts with a system service. [new]

[qinxuye]

Handled the remaining notes from this review in 7d77139:

• uv installation output now streams. A failed first service startup removes the newly created registration; upgrade rollback wording is scoped explicitly.
• Replacement checks exclude only the managed service tree and reject any separate foreground process. An unchanged installation-only repeat succeeds without stopping those processes.
• PowerShell runs in a local script scope; Windows CI invokes it through Invoke-Expression and checks caller variables. macOS graphical-session requirements and console-log retention are documented; no new launchd/log-rotation subsystem is added.
• Existing Xinference commands produce a PATH/environment note. Intel-Mac resolution warns about recent PyTorch wheel availability. README explicitly identifies the base package and optional/on-demand engine dependencies; the pip xinference[all] Quick Start is retained.
• Installer CI follows checkout v7 and read-only contents permissions, exercises real per-platform uv/service behavior, and preserves the original error if cleanup fails.

Local validation: 67 focused tests passed, real offline uv install/upgrade/repeat/downgrade passed, and lint plus English/nine-locale documentation checks passed. The exact-head native matrix passed on Linux, macOS, and Windows, including upgrade, no-op repeat, rollback, downgrade, and native service lifecycle checks. Service-account/ACL redesign, transactional environment relocation, and automatic launchd log rotation remain separate work. Full project CI is tracked independently.

[rogercloud]

Major

• [prior] xinference/deploy/service.py:333 — the Windows LocalSystem service still runs an interpreter from the installing admin's user profile; see the reply on the original thread.

Minor

• scripts/manage_install.py:380 — restore() stops a service that may already be uninstalled, so the installer can get stuck permanently (inline).
• xinference/deploy/service.py:401 — the definition is written before config.json; an interruption leaves an "unmanaged" service that blocks every reinstall (inline).
• xinference/deploy/service.py:333 — --system install --user OTHER never checks that OTHER can execute the invoking user's interpreter (inline).
• [prior] scripts/manage_install.py:467 — on the unchanged early return, configure() runs outside the rollback try, so a newly registered service (e.g. none -> system) whose start fails stays registered, contrary to installation.rst:67 (inline).
• scripts/install.sh:37, scripts/install.ps1:27 — the driver is fetched from mutable main but manages any release (including downgrades) while hardcoding service.py config paths/schema, unit names and flags (scripts/manage_install.py:135-211,247-296); the only gate is 'service' in cli.commands (:446-449). Pin the URL to a release tag or add an explicit compat/version check.
• [prior] Test gaps: xinference/deploy/test/test_installer.py:93-108 still stubs install_tool/active (no argv assertion; Darwin omits --torch-backend) and has no active()/service_pid() parsing tests. Also untested: _processes parsing (test_service.py:51 stubs it), Darwin _control, Windows started/stopped detection, logs(), configure() argv (--home/--user). test_service.py:428 parametrizes 4 modes that install.sh never branches on, and native CI (check_install.py:61 hardcodes --system) never exercises user-mode services.
• scripts/install.ps1:28 — add -UseBasicParsing (inline). xinference/deploy/test/test_installer.py:32 — add a skip guard (inline).
• READMES/README_*.md are not updated with the new README.md installer section.
• scripts/manage_install.py:436-468 — same-version reruns do a full staged install before detecting unchanged; consider a version short-circuit.
• scripts/manage_install.py:522-536 — --constraints is persisted in the uv receipt, so uv tool upgrade xinference silently does nothing; document it.
• scripts/manage_install.py:261-295, xinference/deploy/service.py:414-538 — launchctl print / WinSW stdout parsing depends on human-readable output.
• scripts/manage_install.py:81-99 — xinference service commands don't take the installer lock; a doc note is enough.
• scripts/manage_install.py:225 — the single up-front sudo -v can expire before stop/restore; refresh it right before stopping the service.

Simplification

• xinference/deploy/service.py L202: delete: the macOS-only restart action in _control has a single caller (L434). Inline launchctl kickstart -k there.
  net: -1 lines possible

Blocking: no — recommended event: APPROVE

[qinxuye]

Followed up on the non-blocking notes in the approval review in 2a658b2. The inline replies cover recovery after service removal, interrupted definition writes, unchanged-package startup cleanup, account access requirements, PowerShell 5.1, and installed-distribution test collection.

• Service mode now requires the candidate's explicit installer API version to match the driver. An incompatible candidate is rejected before stop/replacement; native CI adds a real incompatible-wheel check and verifies the running PID is preserved. Downloading the current driver remains supported for latest releases and compatible downgrades.
• Documentation now states the constraints retained by uv tool upgrade xinference, recommends the one-command installer for its managed upgrades, and says manual service commands must not overlap the installer. sudo authentication is refreshed immediately before every service stop, including rollback.
• All nine translated READMEs now contain the installer section. Installation notes, recovery instructions, account access requirements, and Windows default-permission limitations are translated in all nine documentation locales.
• Added regression coverage for the concrete recovery/cleanup failures. Removed the wrapper's redundant four-mode parametrization. Native CI exercises actual service control and rollback; Windows now also uses Windows PowerShell 5.1. Live user-service sessions and additional parser/platform permutations remain outside this matrix; no claim of that coverage is made.
• Kept candidate preparation on same-version repeats, the full backup transaction, and the native CLI control/parsing interfaces. Skipping resolution is a performance follow-up; changing service accounts/ACLs, replacing native interfaces, and adding a user-service matrix are separate work. The macOS kickstart remains in the native control dispatcher.

Validation: 68 focused tests passed; real offline uv install/upgrade/repeat/downgrade/unpin passed; installed-distribution test collection skips cleanly without scripts; lint, msgfmt, catalog compilation, and the installation page in English/all nine locales passed. Exact-head installer CI is running; full project CI is tracked separately.