v6.7.1 into main - #835
Merged
Merged
Conversation
…roject#1288) The previous sed-based approach updated the first `ref:` (and any `repo:`) line in empire/server/config.yaml. After empire_compiler was added to config.yaml above starkiller, the sponsor and public release flows started clobbering empire_compiler.ref/repo instead of starkiller.ref/repo (see PR EmpireProject#1287). Switch to yq for explicit `.starkiller.ref` and `.starkiller.repo` edits. yq is preinstalled on ubuntu-latest runners.
…ct#1309) Increase the request-arrival jitter window from 2s to 5s in the concurrency-250 case. The old window produced ~125 req/s average arrival against an 8-connection pool, which under CI resource contention pushed the queue wait past httpx's 30s timeout (~10% flake rate). 5s halves the avg arrival rate to 50 req/s while keeping the 250 target intact. Refs EmpireProject#966
…roject#1311) * refactor(stager): extract _build_template_vars helper Behavior unchanged. Enables the upcoming go.py race fix to drive generate_main_go from tests with the same template_vars dict the production path uses, without duplicating ~25 lines of construction. * feat(go): generate_main_go returns the rendered string Additive change: file write is unchanged. Enables tests to assert on the exact bytes the renderer produced without reading them off a shared on-disk path. * fix(go): isolate compile_stager builds in per-call temp dirs Each compile_stager call now copies the gopire package into a fresh tempfile.TemporaryDirectory and runs `go build` from there. The canonical install tree is no longer mutated at runtime. Eliminates the race where two concurrent stager-generation requests clobbered each other's main.go between render and build, producing a binary whose embedded values (listener address, staging key, certs) came from the wrong call. The build never errored; the wrong binary was returned silently. The test contract is updated: - test_generate_go_stageless asserts the regex against the rendered string returned by generate_main_go, not a global on-disk file. - test_compile_stager_isolation is added as a post-fix invariant: two sequential calls produce distinct binaries and do not leave a main.go in the canonical install tree. compile_stager's public return type is unchanged (still str). * fix(go): strengthen isolation test and preserve main.go on build failure Addresses two findings from the final code review: - test_compile_stager_isolation now uses distinguishable STAGING_KEY values and asserts the two binaries' sha256 differ, proving cross-contamination cannot happen. The previous version only compared paths, which proved the random-suffix function worked but did not catch a hypothetical regression where compile_stager returned cached content regardless of template_vars. - On go build failure, copy the rendered main.go to a stable gopire-failed-main-*.go file outside the TemporaryDirectory and reference its path in the exception message, so the operator can inspect the file the build error refers to. The TemporaryDirectory cleanup otherwise removes the file before the exception propagates. * fix(go): harden compile_stager error-path lifecycle + concurrent isolation test Three review-driven fixes on top of the static-path race fix: 1. Eliminate orphan temp file on every build failure path. Restructure compile_stager to keep the build output inside the per-call TemporaryDirectory and shutil.move it out on success only. The previous NamedTemporaryFile(delete=False) placeholder leaked an empty file into $TMPDIR on every copytree/jinja/subprocess/build failure. 2. Never let preservation displace the real build error. mkstemp / copy2 can fail (ENOSPC, EACCES, race on main.go); previously such failures would mask the actual `go build` stderr the user needs. Wrap in try/except, log on preservation failure, and append a "failed to preserve" note — the ModuleExecutionException carrying the build stderr always reaches the caller. 3. Replace sequential isolation test with concurrent variant. The previous test_compile_stager_isolation ran two builds back-to-back; sha256-distinct binaries would have passed under the buggy pre-fix code too. The new test_compile_stager_concurrent_isolation submits four compile_stager calls to a ThreadPoolExecutor and asserts each binary contains its own staging key as a literal byte string — catches actual cross-contamination under the race the fix targets. Also adds two new tests for the previously-untested build-failure preservation branch (verifies preserved-file path in message + survival of TemporaryDirectory cleanup, and verifies build stderr propagates when preservation itself fails). Converts test cleanup `unlink(missing_ok=True)` of the canonical install tree to precondition `assert not exists()` so test pollution fails loudly instead of being silently masked. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6 to 7. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](docker/build-push-action@v6...v7) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ect#1190) Bumps [MishaKav/pytest-coverage-comment](https://github.com/mishakav/pytest-coverage-comment) from 1.2.0 to 1.6.0. - [Release notes](https://github.com/mishakav/pytest-coverage-comment/releases) - [Changelog](https://github.com/MishaKav/pytest-coverage-comment/blob/main/CHANGELOG.md) - [Commits](MishaKav/pytest-coverage-comment@v1.2.0...v1.6.0) --- updated-dependencies: - dependency-name: MishaKav/pytest-coverage-comment dependency-version: 1.6.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 47.0.1 to 47.0.5. - [Release notes](https://github.com/tj-actions/changed-files/releases) - [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md) - [Commits](tj-actions/changed-files@v47.0.1...v47.0.5) --- updated-dependencies: - dependency-name: tj-actions/changed-files dependency-version: 47.0.5 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](docker/setup-buildx-action@v3...v4) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4. - [Release notes](https://github.com/docker/setup-qemu-action/releases) - [Commits](docker/setup-qemu-action@v3...v4) --- updated-dependencies: - dependency-name: docker/setup-qemu-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…to destination (EmpireProject#1322) * fix(git-util): clean up /tmp staging dir after clone_git_repo copies to destination clone_git_repo always stages clones in Path(tempfile.gettempdir()) / random_string(5) before copying them to a caller-supplied directory (used so the clone can run as the sudo invoker via run_as_user, picking up their SSH agent for private-repo access). The staging dir was never removed, so every call leaked ~30 MB to /tmp. Tests wipe DATA_DIR each session, which forces sync_starkiller to re-clone, so on dev machines with a tmpfs-backed /tmp the leak accumulated until the filesystem filled up. A full /tmp produces a cascade of unrelated-looking test failures (pytest temp fixtures, MySQL temp tables, and Empire's own stager builds all silently fail), e.g. "Cannot execute disabled module", "Plugin not found in marketplace", 400 != 201. CI never reproduces because each runner starts with a fresh /tmp. Fix: wrap the body in try/finally and rmtree the staging dir, unless directory=None (the caller-owns-the-tmp-dir path used by install_plugin_from_git), in which case ownership is transferred by nulling the local before returning. Partial staging dirs from failed clone/checkout are always cleaned. Adds empire/test/test_git_util.py covering all four paths (directory=set success, directory=None success, clone failure, checkout failure) using a monkeypatched run_as_user so the tests don't shell out to git. * test(git-util): tighten test isolation and close coverage gaps Review-feedback follow-up to the clone_git_repo cleanup fix: - Fix pre-existing docstring :param order — `directory` was listed before `ref` but the signature is (git_url, ref, directory). Spotted while reviewing the new docstring text. - Isolate the suite from the host /tmp via an `isolated_tempdir` fixture that monkeypatches `git_util.tempfile.gettempdir` to point at pytest's per-test tmp_path. Replaces the brittle 5-char-name heuristic in `_list_tmp_clone_dirs`; the magic number `5` no longer leaks from production into the test file. - Promote the fake `run_as_user` to a `_FakeRunAsUser` class that tracks the last-cloned destination and asserts `cwd == clone_dest` on every checkout — catches signature drift on `run_as_user(cwd=...)` that the previous fake would have silently dropped. - Add `test_clone_with_directory_and_ref_success` covering the clone_ok + checkout_ok + directory= path that no previous test exercised. A broken `cwd` wiring would have slipped through. - Add `test_copytree_failure_cleans_tmp_dir` covering the shutil.copytree failure branch — verifies the new `finally` block cleans the staging dir even when the copy itself raises. New ("copytree_destroys_clone",) behavior in the fake removes the staging dir immediately after creation so copytree hits a FileNotFoundError. * test(git-util): harden fake run_as_user + close last coverage gaps Second-pass review feedback follow-up: - _FakeRunAsUser.__call__ now mirrors run_as_user's real signature (user, cwd, capture_output, check, text) and asserts on unknown kwargs, so production-side signature drift surfaces as a loud test failure instead of being silently swallowed by **kwargs. - Add a precondition assertion to the checkout_ok branch (last_clone_dest is not None) so scripting "checkout_ok" without a prior "clone_ok" fails with a clear "test-script bug" message instead of a confusing PosixPath != None mismatch. - Add test_clone_without_directory_with_ref_success covering the clone_ok + checkout_ok + directory=None combo. Guards against a refactor that clears tmp_dir = None before the checkout block, which would silently leak the staging dir on success. - Rewrite test_copytree_failure_cleans_tmp_dir to trigger a real-world failure mode (pre-existing destination → FileExistsError from shutil.copytree) instead of the previous contrived "destroy source mid-flight → FileNotFoundError." Also drops the now-unused copytree_destroys_clone behavior token. - Broaden _staging_dirs (renamed _leftovers) to include files, not just directories, so a future bug leaking a non-directory artifact next to the staging area trips the assertion. - Rephrase clone_git_repo docstring for precision: "staged in a directory under the system tempdir" (was: "tempfile path"), "lifecycle" (was: "ownership of the staging path") to avoid implying filesystem ownership transfer. - Document the checkout_ok precondition in the _FakeRunAsUser docstring.
…es (EmpireProject#1185) * fix: remove serializing lock from AgentCommunicationService file writes The single instance-level threading.Lock was serializing all concurrent file downloads from all agents through a single bottleneck. Each agent writes to its own isolated session_id/ subdirectory, so there is no shared mutable state requiring a lock. - _is_path_safe is a pure stateless Path.resolve() comparison - mkdir(parents=True, exist_ok=True) is thread-safe and idempotent - Agents write to separate per-agent directories with no cross-agent conflicts Closes EmpireProject#1182 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: address file handle leak and ZeroDivisionError in save_file - Use `with` block for file writes to prevent handle leaks if decompression raises an exception - Guard percentage calculation against ZeroDivisionError/ValueError from agent-supplied total_filesize Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: catch OSError on save_file.stat() and enrich progress warning log - Wrap save_file.stat() inside the percent-calc try/except so a filesystem hiccup between the write completing and the stat call doesn't propagate to the outer except Exception at _handle_agent_response and turn a successful download into a generic "Error processing result packet". - Enrich the warning log with tasking.id and bytes_written so an upstream agent sending bad total_filesize on every chunk can be correlated to the specific task and confirmed against bytes actually on disk. - Add CHANGELOG entry under [Unreleased] > Fixed covering the lock removal plus the file-handle and percent-calc fixes from a98b37a. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…mpireProject#1328) Three independent fixes for stager compile/generation error paths that have been swallowing diagnostic detail and producing unactionable error messages. go.py compile_stager + dotnet.py compile_task/compile_stager: - include result.returncode in the raised ModuleExecutionException message and in a log.error before the raise - fall back through stderr -> stdout -> a signal-kill hint when stderr is empty. An empty stderr with rc != 0 is the fingerprint of a process killed by signal (OOM-killer, cgroup limit, AV) and the previous message ("Go build failed: ", "EmpireCompiler execution failed with error: ") gave the operator no way to tell. This is the failure mode reported in PR EmpireProject#1306 review by Anthony — operator could not distinguish OOM from a real go-build error from the message alone go.py compile_stager: - swap env merge order so explicit function args (goos/goarch) win over operator-set GOOS/GOARCH. Previous {**env, **os.environ} let an operator with GOOS=linux in their shell silently override a function-arg goos="windows" cross-compile stager_service.generate_stager: - catch ModuleExecutionException from template_instance.generate() and return structured (None, str(e)) so the API surfaces a 400 with the message instead of a 500 stack trace. Log with exc_info=True so the chained subprocess context survives for empire-side debugging Tests: - new empire/test/test_dotnet.py with parametrized rc!=0 diagnostic tests for compile_task and compile_stager (signal-kill / stderr / stdout-fallback cases) - empire/test/test_go.py gains the same parametrized diagnostic test for compile_stager, plus an env-merge ordering test pinning function-arg precedence over operator env - empire/test/test_stager_api.py gains a unit test asserting generate_stager catches ModuleExecutionException, returns the message, and logs at ERROR with exc_info=True All new tests pass. Pre-existing local-env failures (test_windows_c_stager_*, test_install_plugin_*) are unrelated to these changes. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…olchain download, autoescape) (EmpireProject#1335) * fix(gopire): resolve go build failures caused by goenv shim, root GOCACHE, and toolchain download - Add _resolve_go_binary() to bypass goenv bash shim (E2BIG/rc=126 when sudo -E carries a large environment into the uvicorn worker); logs a warning on every fallback path instead of silently swallowing errors - Fix compile_stager: GOCACHE now uses install_path-relative directory so root-run server does not get a cold cache on every restart; add GOTOOLCHAIN=local to prevent phantom toolchain downloads on Go 1.21+; fix env merge order so function-arg goos/goarch win over operator shell exports; wrap subprocess.run in try/except so FileNotFoundError becomes a ModuleExecutionException (400) rather than an unhandled 500 - Fix Jinja2 autoescape=True → False so '&' in profile URIs is not HTML-escaped to '&' in generated Go source - Remove dead compile_task method (referenced self.compiler, never defined) - Fix go.mod toolchain directive typo (go1.23.10 → go1.23.1) - Add vendor/ directory for offline builds (no network I/O during go build) - Catch ModuleExecutionException in stager_service.generate_stager so compile failures return 400 Bad Request instead of 500 Internal Server Error; guard post-generation file I/O with its own OSError handler - Rewrite test_go.py with 19 tests covering all fixed bugs; fix mock fragility in test_stager_generation_service.py; add 400 API test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: remove gopire vendor directory from tracking vendor/ should not be committed — add it to .gitignore and remove the previously staged files from the index. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: apply ruff format to test_go.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: apply ruff format to test_stager_generation_service.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Four unrelated, surgical modernization fixes: - jwt_auth: replace deprecated datetime.utcnow() with timezone-aware datetime.now(UTC) in create_access_token. - core/db/base: replace five O(n) len(db.query(X).all()) == 0 startup existence checks with O(1) db.query(X).first() is None. - common/helpers: drop the Python-2 old_div shim from get_file_size (// is equivalent for int operands) and delete the now-unused empire/server/utils/math_util.py; agent-payload copies under data/agent/ left untouched. Also corrects the GB-size calculation (Copilot Autofix). - pre-commit: bump ruff-pre-commit rev v0.12.12 -> v0.15.2 to match the pyproject dev-dep pin.
…#1340) * Migrate ORM models to SQLAlchemy 2.0 typed declarative Convert empire/server/core/db/models.py from the legacy 1.x style (Column() + declarative_base()) to the 2.0 typed declarative style (Mapped[] + mapped_column() + DeclarativeBase). This is a schema-preserving change: the generated DDL (verified for both the sqlite and mysql dialects, including indexes and the with_variant LONGTEXT columns) is byte-for-byte identical before and after. Column nullability is derived from each Column's nullable= state and reflected in the Mapped[T] / Mapped[T | None] annotations. The association Table constructs, hybrid_property/expression definitions, and relationship loading strategies are unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Add changelog entry for SQLAlchemy 2.0 models migration Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EmpireProject#1338) * refactor(db): migrate core data services to SQLAlchemy 2.0 select() API Behavior-preserving migration of the legacy 1.x Query API (db.query(...).filter(...)) to the 2.0 select() style across the core service layer. Sessions remain synchronous (SessionLocal / Session); this is a syntax/idiom change only. - .filter().first()/.all() -> db.scalars(select().where()).first()/.all() - single-column selects -> db.scalar()/db.scalars() (values, not Row tuples) - multi-column window-count query builders (get_tasks, get_agent_checkins, tag/download get_all) -> db.execute(select(...)).all() to keep Row access - bulk .delete()/.update() -> db.execute(delete()/update()) - func.max scalar, len(query.all())==0 count checks -> select(func.count()) - == True / == False (noqa: E712) filters -> .is_(True)/.is_(False) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(db): migrate api/utils/listener/plugin call sites to SQLAlchemy 2.0 select() Behavior-preserving 1.x->2.0 query migration for the remaining non-core db.query(...) call sites: - api/jwt_auth.py, api/v2/websocket/socketio.py - utils/data_util.py (get_listener_options keeps returning a Row via db.execute(...).first() to preserve existing caller behavior) - listeners/http.py, listeners/http_malleable.py (chained SessionLocal().query()) - plugins/basic_reporting Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: update mocked DB call chains for SQLAlchemy 2.0 select() migration These unit tests monkeypatch SessionLocal and assert against a mocked db.query(...).filter(...).first() chain. The core services they exercise (BypassService, ProfileService, http_malleable validate_options) now call db.scalars(select(...).where(...)).first(), so the mocks must target .scalars().first() instead. Production behavior is unchanged; only the mock wiring is updated to match the new call pattern. Note: test_profile_loader's mock is fixed for parity even though it is currently masked locally by an empty profiles submodule (it would fail in CI once the submodule is populated). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(hooks): update autobegin warning to 2.0 query idiom The _run_async_hook docstring used db.query() as its concrete example of a call that triggers SQLAlchemy autobegin. After the 1.x->2.0 migration that idiom no longer appears anywhere in the server, so the example now points at the one pattern maintainers won't write. Generalize it to db.scalars(select(...)) / db.execute(...). Comment-only change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(db): simplify existence-check selects to db.scalar() Follow-up polish from code review. Where a migrated select's result is used only for existence/truthiness or is discarded, collapse db.scalars(select(...)).first() to db.scalar(select(...)) — same first-or-None contract, matching the db.scalar() idiom already used elsewhere in this PR. - credential_service.check_duplicate_credential: `found is not None` - plugin_registry_service.load_plugin_registries: truthiness `if ...: continue` - db/base.startup_db: discarded schema smoke-test query bypass_service.load_bypasses was intentionally left as db.scalars(...).first() so its existence check stays consistent with the adjacent fetch-and-mutate call in the same loop. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(changelog): add SQLAlchemy 2.0 query-migration entry Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(changelog): condense Unreleased entries to match prior style The Unreleased entries had grown into 4-8 sentence paragraphs. Trimmed each to the one-sentence (occasionally two) form used by older releases, preserving the key fact of every entry. Also placed the two SQLAlchemy migrations (query API + typed-declarative models) adjacent under Changed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](docker/login-action@v3...v4) --- updated-dependencies: - dependency-name: docker/login-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Removed duplicate version entry for 6.5.0.
EmpireProject#1345) The shutil.rmtree(DATA_DIR) at config_manager.py:321-322 ran as a module-import side-effect any time TEST_MODE was set in env. The perf-test fixture in empire/test/test_performance/conftest.py spawns a `poetry run python empire.py server` subprocess; that subprocess inherits TEST_MODE=true from pytest.ini, imports config_manager fresh, and wipes the EmpireCompiler cache the parent pytest process just spent ~27 s downloading. Move the wipe + registry-reseed into a new _reset_test_dirs() helper called from root conftest.py's pytest_configure. The wipe now fires exactly once per pytest session, before any test or fixture runs, and crucially before config_manager is imported anywhere. Subsequent config_manager imports (in the perf subprocess, in test modules) hit the existing idempotent mkdir + DEFAULT_CONFIG fallthrough at config_manager.py:333-345 and reuse the cached compiler. Adds empire/test/test_config_manager.py with a regression test that asserts importing config_manager in a TEST_MODE subprocess does NOT delete pre-existing DATA_DIR contents. Measured impact on empire/test/test_performance/test_blocking.py::test_stager_creation_does_not_block_management_api: - empire_base_url fixture setup: 35.7 s -> ~8.4 s - total setup phase: 84.2 s -> ~48.5 s The duplicated `~/.local/share/empire-test` / `~/.config/empire-test` path literals between conftest.py and config_manager.py are deliberate and have bilateral sync comments — importing the paths from config_manager would trigger a premature module-body execution that builds empire_config from the production default config instead of test_server_config.yaml. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ct#1343) * chore(deps): drop 4 unused/redundant Python dependencies Removes packages with no remaining first-party imports plus one direct pin that pytest-cov already pulls transitively. Verified via import grep across empire/server, empire/test, setup/, and empire.py, and confirmed each was either vestigial from a completed migration or orphaned when the only consumer changed. - pycryptodome: superseded by `cryptography` in the 2021 pycrypto migration (#92); no `Crypto.*` imports remain anywhere, including data/, stagers/, and modules/. - pyOpenSSL: in initial 2020 pyproject; last `import OpenSSL` removed in 2020 "Removed unused imports" (#241), but the pin was forgotten. - pytest-timeout (dev): added 2022 for `@pytest.mark.timeout(30)` on two `test_zz_reset` cases; those markers no longer exist. - coverage[toml] (dev): redundant direct pin — pytest-cov already requires `coverage[toml] >= 7.10.6`. `urllib3` was considered but kept: it's used in `server.py` to suppress `InsecureRequestWarning`, so the direct pin stays. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(changelog): move Removed block to match Keep-a-Changelog order Repo convention (per the header comment lines 8-13 and every prior release block) is Added → Changed → Removed → Fixed. Move the new Removed entry to sit between Changed and Fixed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Use FromBase64String instead of NSSBase64_DecodeBuffer * Dump credentials from all Firefox profiles and remove loading of unused dlls * Ensure x86/x64 compatibility * Fix indentation * Add entry to CHANGELOG.md --------- Co-authored-by: Anthony Rose <20302208+Cx01N@users.noreply.github.com>
Pre-commit hooks (trailing-whitespace + end-of-file-fixer) auto-fixed Get-FoxDump.ps1 — same lint fix being applied upstream on #823 (release/6.6.0 → main). Folded into this cherry-pick so EmpireProject#1352 lands clean rather than landing dirty and needing a follow-up. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* Fix powershell foxdump (#812) * Use FromBase64String instead of NSSBase64_DecodeBuffer * Dump credentials from all Firefox profiles and remove loading of unused dlls * Ensure x86/x64 compatibility * Fix indentation * Add entry to CHANGELOG.md --------- Co-authored-by: Anthony Rose <20302208+Cx01N@users.noreply.github.com> * style(foxdump): strip trailing whitespace + single trailing newline Pre-commit hooks (trailing-whitespace + end-of-file-fixer) auto-fixed Get-FoxDump.ps1 — same lint fix being applied upstream on #823 (release/6.6.0 → main). Folded into this cherry-pick so EmpireProject#1352 lands clean rather than landing dirty and needing a follow-up. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Andrei SÂNTOMA <andreisantoma@gmail.com> Co-authored-by: Anthony Rose <20302208+Cx01N@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
…roject#1346) * chore(deps): drop requests-file by inlining file:// dispatch `plugin_service` mounted `requests_file.FileAdapter` on a `requests.Session` solely so that `s.get("file:///path")` worked in one method (`_download_tar`). Replaced with explicit scheme handling: `urllib.parse.urlparse` detects the `file://` URLs, then `Path.open("rb")` + `tarfile.open(fileobj=...)`. The HTTP download path now uses `requests.get(...)` directly. Net install-set: -1 (requests-file). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(plugin_service): wrap file:// open in PluginValidationException The pre-refactor `requests-file` adapter returned a non-200 Response when the target file was missing, which the existing `response.status_code != HTTP_200_OK` check converted into a `PluginValidationException` → 400. After the refactor, the file:// branch called `Path.open("rb")` directly, so a missing file surfaced as an unhandled `FileNotFoundError` → 500. Caught by `test_install_plugin_tar_invalid` (and the mysql CI leg on PR EmpireProject#1346) which asserts a 400 response body of `"Failed to download plugin: <OSError str>"`. Wraps the file:// open in `try/except OSError` and re-raises as `PluginValidationException(f"Failed to download plugin: {e}")` — matches the prior error message format byte-for-byte. Verified locally: - Invalid path raises `PluginValidationException("Failed to download plugin: [Errno 2] No such file or directory: '/some/invalid/tar/url'")` — exact match for the test assertion. - Valid tar still extracts successfully. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…se (EmpireProject#1302) When an agent posted back a batch of N result packets, the response handler was acquiring N pool connections and opening N database transactions — one per packet — even though all the work touches the same agent. Hoist `SessionLocal.begin()` outside the per-packet loop. One session, one connection, one transaction per agent callback. As a side effect: - `update_agent_lastseen` runs at most once instead of N times. - `AFTER_TASKING_RESULT_HOOK` still fires per-tasking but is now collected and iterated outside the session block (preserving the prior "fire hooks outside session" invariant). Trade-off (worth calling out for review): the prior code committed each result individually, so a failure mid-batch left earlier results persisted. The new code is all-or-nothing per callback. For result-packet processing this is the more sensible default — agents retry whole batches on connection failure, and partial commits under DB error already weren't reliable since the tasks share an agent and listener. Local: test_agent_api.py + test_agent_task_api.py + test_agents.py + test_agent_communication_service.py — 104 passed. Production impact: a typical 5-result callback drops from 5 connection acquisitions + 5 BEGIN/COMMIT pairs to 1 of each. Reduces pool pressure under high agent load.
* db: index FK + auth-lookup columns; composite index for listener-agent query
Adds four indexes that target the common WHERE patterns identified
in the phase-2 perf audit:
- users.username: auth lookups by username (auth
is on every API request)
- agent_files.session_id: file-tree ops filter by session
(per-agent file enumeration)
- agent_files.parent_id: tree traversal queries
- agents (listener, archived): AgentService.get_for_listener
composite WHERE listener=?
AND archived=false
Declared as `index=True` on the model columns and as a composite
`Index()` in `Agent.__table_args__`. New databases pick these up
through `Base.metadata.create_all()` at startup_db time.
Existing databases that were stamped at the 0001 baseline before
these declarations existed will NOT have the indexes until a
follow-up migration is applied. Adding that migration cleanly
requires refactoring `test_alembic.py`'s synthetic-migration
fixtures off the hard-coded "0002" revision id (they currently
collide with any newly-added real migration). That refactor is
deliberately out of scope for this commit; tracked separately.
Verified locally: SHOW INDEXES on a freshly-created test_empire DB
shows all four indexes after this commit.
* perf: shorten boot path on listener startup + profile preload
Two boot-path cleanups identified by the phase-2 audit.
1. Listener.start() — drop the 1s post-thread-start sleep on the
prod path. The sleep existed so .is_alive() could return a
useful "did the thread crash on startup?" signal. Replaced with
`self.thread.join(0.1)`, which has the same correctness signal
(thread is either still alive at 100ms or it already exited)
and bails out as soon as the thread dies. Cuts ~0.9s × N
listeners off prod restart.
Applied in three places that all duplicate the same pattern:
- listeners/template.py (the base)
- listeners/http.py (overrides)
- listeners/http_malleable.py (overrides)
Also drops now-unused `time` and `os` imports from template.py.
2. profile_service.load_malleable_profiles — pre-load existing
profile names with a single `SELECT name FROM profiles` instead
of issuing a `SELECT...WHERE name=?` per .profile file in the
rglob loop. Mirrors the pattern already in
`module_service.load_modules`. ~76 .profile files; saves ~75
redundant queries on first boot of a fresh DB.
Local: test_listener_api.py + test_profile_api.py — 36 tests
passed.
* perf: lazy-load custom_generate module classes on first use
`module_service.load_modules` was eagerly importing + instantiating
the `Module` class for every module with `custom_generate: true`
(~130 modules in the in-tree set). Each one runs:
- importlib.util.spec_from_file_location
- module_from_spec
- exec_module (executes the .py at the top level)
- Module()
`generate_class` is only consumed in `_generate_script` (line ~382),
so eager-loading at boot is wasted work for the >90% of test sessions
and production restarts that never invoke a custom_generate module.
Refactor:
- Add `custom_generate_path: str | None = None` to
EmpireModuleAdvanced. At boot, store the .py path; don't import.
- Add `ModuleService._load_custom_generate_class(module)` which
imports + instantiates on first access and caches the result on
`module.advanced.generate_class`.
- `_generate_script` calls the helper; subsequent invocations reuse
the cached class. Existing `module.advanced.generate_class`
attribute is preserved for backwards compatibility.
The existence check at boot (`custom_py.exists()`) is kept so a
missing .py still surfaces immediately at MainMenu init rather than
at first module execution.
Local: test_modules.py + test_module_service.py + module-related
test_agent_task_api.py — 101 tests passed.
* docs: update CHANGELOG for production-quickwins cluster
* fix: keep listener startup grace at 1s in production path
The earlier change replaced time.sleep with self.thread.join(0.1) in
all paths, which dropped the prod-mode startup grace from 1s to 100ms.
Under TEST_MODE the original sleep was already 0.1s so the test path
was unaffected, but the perf-test suite spawns Empire as a real
subprocess without TEST_MODE — so its listener was getting 100ms of
grace before tests started hammering it. Caused intermittent failure
of test_pool_exhaustion[250] (all 250 connections timing out because
the listener hadn't bound its port yet).
Restoring the original timeout split: 0.1s under TEST_MODE, 1.0s in
prod. join() still gives us the early-exit-on-thread-death speedup
that motivated the original change — that benefit is on the crash
path only, the success path waits the full timeout either way.
* review: tighten error paths + fix CHANGELOG wording
- CHANGELOG: AgentFile.session_id is a logical reference, not a declared
FK (only parent_id is). Reword the indexes line accordingly.
- module_service._load_custom_generate_class: catch import / instantiation
failures and re-raise as ModuleValidationException so a broken
custom_generate .py surfaces as a specific user-facing 400 ("failed to
load <path>: <error>") instead of the generic "Error generating
script." fallback. Also add an explicit `spec is None or spec.loader
is None` guard with a clearer message than the AttributeError that
spec.loader.exec_module would otherwise raise.
- profile_service.load_malleable_profiles: distinguish "already in DB"
(silent skip, normal reload) from "duplicate filename in this load
pass" (log.warning, surfaces misnamed files to operators).
Follow-up issue EmpireProject#1310 covers converting AgentFile.session_id to a real
FK with the matching String(255) widening once the alembic-fixture
conflict is resolved.
* review: tests + comment cleanup from PR feedback
- Add boot-time sweep in test_load_modules so a broken in-tree
custom_generate .py still fails CI (the lazy-load defer would
otherwise mask import errors until first execute).
- Add unit tests for _load_custom_generate_class: cache hit,
caches-after-first-call, None-spec wrap, and exec_module-error
wrap (with __cause__ preserved).
- Trim/rework comments at risk of rotting: drop the perf-test
suite name from template.py, drop the "130+ modules" count
from module_models.py, replace the inline SQL on the agents
composite index with a name-only call-site cite, replace the
http.py / http_malleable.py cross-refs to template.py with a
one-line WHY.
- Add WHY-comments on the User.username and AgentFile.session_id
/ parent_id single-column indexes naming the access path.
- Drop the dead (ModuleValidationException, ModuleExecutionException)
re-raise from _load_custom_generate_class — Module.__init__ is
not contracted to raise either; the bare Exception branch is
enough.
- Add a docstring note that _load_custom_generate_class is
intentionally not thread-safe (benign for in-tree modules).
- Profile dup warning logs first-occurrence path too.
- Add test_profile_loader_warns_on_duplicate; align the existing
test_profile_loader mock to the new query(Profile.name).all()
call shape.
* revert: drop Thread.join listener micro-opt
The Thread.join(1.0) swap only saves ~1s on the rare crash path
(port-in-use, bind error). With production keeping the full 1s
bind cushion, the success path is unchanged. Not worth the
explanatory comments and the cognitive cost of "why join not
sleep" — revert to time.sleep across template.py, http.py,
http_malleable.py. Profile preload (the other half of the
boot-path quickwin) stays.
…ireProject#1355) * fix(security): use constant-time comparison in verify_hmac The double-HMAC in AESCipher.verify_hmac compared digests with `==`, which is vulnerable to timing attacks. Switch to hmac.compare_digest via the already-imported ct_compare_digest alias. * refactor: remove unused random_provider constant The `random_provider = "Python SSL"` module-level constant in encryption.py was never read anywhere in the codebase. * fix(auth): require enabled user for protected endpoints (CurrentActiveUser) `CurrentUser` resolves via `get_current_user_from_token`, which does not re-check `user.enabled`. The intended dependency for protected endpoints is `CurrentActiveUser` (uses `get_current_active_user`, which rejects disabled users with 400). Both `agent_task_api.py` and `plugin_api.py` were applying `get_current_active_user` as a router-level dependency, so disabled users were already gated at the router; this change brings the per-route parameter annotation in line with that intent so the safety holds even if the router-level dep is ever removed or the endpoint is copied elsewhere. The unscoped `CurrentUser` alias in `jwt_auth.py` is retained because it is referenced internally by `get_current_active_user` and `get_current_active_admin_user`, both of which apply their own `.enabled` check. * fix(api): remove duplicate fields from CredentialUpdateRequest `os`, `sid`, and `notes` were declared twice — first as required `str`, then as `str | None = None`. The second declarations were silently overriding the first at class-body evaluation. Drop the required versions; keep the optional ones (which match CredentialPostRequest). * fix(api): guard total_pages against non-positive limit `math.ceil(total / limit)` produces a negative result when `limit` is the project-wide "unbounded" sentinel of -1, and would raise on `limit=0`. Four endpoints had inline `if limit > 0 else page` guards already; four (agent_api, agent_task_api, plugin_task_api) did not. Extract a `paginate(total, page, limit) -> tuple[int, int]` helper in `shared_dependencies` so every paginated endpoint computes the same metadata. For non-positive `limit`, the response collapses to a single page with `total_pages` mirroring the requested `page` (matching the existing guarded callsites). Migrate all eight callsites (agent_api, agent_task_api x2, plugin_task_api x2, download_api, tag_api x1). * fix(api): paginate normalizes to (1, ...) for non-positive limit `paginate(total, page, limit)` previously returned `(page, page)` when `limit <= 0`, which produced a misleading response shape: `?page=3&limit=-1` reported `total_pages=3`, falsely implying pages 1 and 2 existed with data. Return `(1, 1 if total > 0 else 0)` instead — the caller's `page` is normalized to `1` and `total_pages` honestly reflects whether any rows exist. Updates the tests accordingly and adds an empty-result + unbounded case. * test(api): pin paginate page pass-through; tighten docstring (review polish) - Add a test asserting paginate() echoes a non-1 page for a positive limit (the existing positive-limit cases all used page=1 and couldn't catch a regression that hardcoded the page). - Tighten the paginate docstring's redundant "non-positive" wording. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(auth): reject disabled users on socket.io connect F007 switched the REST endpoints to CurrentActiveUser so a disabled user's still-valid token is rejected, but the socket.io connect handler authenticated via get_current_user_from_token, which validates the token without checking account state. A user disabled mid-session could still open a socket and receive the realtime event stream (agent callbacks, task output, new listeners) until their token expired. Mirror the get_current_active_user .enabled check in get_user_from_token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…reProject#1337) * fix(security): apply real bandit (S) findings from ruff triage Three flake8-bandit findings were genuine and worth fixing rather than suppressing: - module_service: the yaml import falls back to the unsafe full Loader/Dumper when libyaml is unavailable, while the primary path uses CSafeLoader. Make the fallback use SafeLoader/SafeDumper so module YAML is always loaded safely (S506). - db/defaults: generate the staging key with secrets.choice (CSPRNG) instead of random.choices. The staging key is agent<->server crypto material (S311). - data_manager: add a 30s timeout to the Empire-compiler download (S113). The adjacent tarfile.extractall is annotated noqa S202 (extracts the Empire-published compiler archive from a configured release URL). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(lint): enable flake8-bandit (S) with C2-aware suppressions Empire is an offensive-security C2 framework, so a large share of bandit findings are intentional by design rather than vulnerabilities. Enable the S rule group and suppress the inherent findings with documented rationale, scoped via per-file-ignores where possible: - listeners/* : S104 (bind-all), S108 (temp), S311 (non-crypto RNG) - stagers/* : S108, S311, S602/S603/S607 (compiler subprocess) - modules/* : S108, S110, S311, S602, S608 (SQL baked into payloads) - core/utils orchestration files: scoped S602/S603/S607/S311/S104/S608 for the subprocess/RNG/bind-all/SQL code that drives payload generation Inline noqa with justification for the one-off security-sensitive cases: - config_manager S105: documented default admin password (overridden in config) - listener_service S324: md5 derives a protocol-locked 32-char staging key - templating S701: jinja2 renders payloads/scripts, not HTML - plugin_service S202: extracts plugin archives from configured registry URLs - db/defaults S311: obfuscation keyword alias, not crypto material modules/* also ignores TRY301 (untested payload-gen code, risky to refactor). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lint): enable tryceratops (TRY) and clean up exception handling Enable the TRY rule group and bump the pre-commit ruff rev to v0.15.2 to match the poetry dev-dependency. Configuration: - ignore TRY003 (raise-vanilla-args, ~230 hits) and TRY002 (raise-vanilla- class, 8 hits): requiring a custom exception class for every raise is too invasive for this codebase and can be a separate effort. Code changes: - TRY400: use log.exception instead of log.error inside except blocks so tracebacks are captured. Where a block already logged the exception (or stacked several log calls), consolidated to a single traceback and demoted supplementary guidance to log.info. - TRY401: dropped the now-redundant exception object from those log.exception messages (and the unused `as e` bindings). - TRY201: replaced `raise e` with bare `raise`. - TRY300: moved trailing success statements into else blocks (kept the risky call inside try so it stays guarded). - TRY301: refactored the create_task_module handler to check the error after the try instead of raising-then-catching an HTTPException. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(lint): address self-review feedback Refinements from a multi-agent review pass; no behavioral change to the lint enablement: - pyproject: correct two per-file-ignore rationales. agent_communication_service S104 is a default client_ip="0.0.0.0" placeholder (not a socket bind), and helpers.py S110 is best-effort parsing of untrusted Mimikatz output (not agent-side payload logic) — gave each its own accurate note. - data_manager / plugin_service S202 noqa: note that filter="data" (py3.12+) would add path-traversal defense-in-depth; suppression stands on source trust. - bypass_service: include the file path in the load-failure log so operators know which bypass YAML failed. - file_util: log a failed command's captured stdout/stderr at warning (not info) so the failure output stays visible under a WARNING+ log config. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(lint): triage new S findings from private-main merge Merging private-main brought in code that the newly-enabled S rules flag: - go.py GoCompiler: jinja2 autoescape=False (S701) — renders Go payload source, not HTML; same rationale as common/templating.py. - plugin_service.py file:// branch (added in EmpireProject#1346): tarfile.extractall (S202) gets the same noqa as the requests branch; requests.get gains timeout=30 to match the data_manager convention (S113). ruff check . and ruff format --check . pass cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(security): use secrets for server-side RNG; tempfile for build dirs Tighten the S311/S108 triage after review feedback: rather than blanket- suppressing non-crypto RNG and shared /tmp paths across the server, fix the server/orchestration layer outright and reserve the suppressions for the payload-generation code where they actually belong. S311 (random -> secrets, CSPRNG) on server-side code, and drop the now-unneeded per-file S311 ignores: - helpers.random_string / string_util.get_random_string — used for protocol nonces and naming; no seeding/reproducibility need, generated server-side. - agent_communication_service.generate_sessionid — the 8-char agent session id. - listener_util.generate_random_cipher, stager_generation_service URI pick. - globentropy (obfuscation seed) keeps an inline noqa; it is cosmetic. S108 (stager_generation_service): the macOS .app launcher build used hardcoded /tmp/application + /tmp/launcher.zip (world-writable, predictable, races between concurrent builds). Switched to tempfile.mkdtemp() (0700) with try/finally cleanup. Dropped its S108 ignore. The listeners/stagers/modules blanket S311/S108 ignores remain: that RNG is inside emitted agent payloads (runs on the victim, not the server) or cosmetic operational choices, and those untested payload-gen files are out of scope to refactor. Updated the pyproject rationale comments to say exactly that. ruff check . and ruff format --check . pass cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(security): harden tar extraction with filter="data" (S202) Audited every S/TRY suppression. Most hold up (option-default placeholders, emitted victim-side payload code, intrinsic compiler spawns with static argv, tests), but the three tarfile.extractall calls were genuinely worth fixing rather than suppressing: - data_manager.py: EmpireCompiler release archive - plugin_service.py (x2): third-party plugin archives (file:// and registry URL) Add filter="data" (Python 3.12+; project floor is 3.13) to defend against path traversal / unsafe members. Verified it works with the streaming r|gz / r|* modes these use. This also removes the S202 finding outright, so the three noqa suppressions are dropped. Also documented obfuscation_service.py's shell=True (S602): its only non-static input is the authenticated operator's obfuscation config, so it is trusted — noted so a future lower-trust caller is recognized as command injection. ruff check . and ruff format --check . pass cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(lint): condense ruff per-file-ignore comments The rationale comments had grown verbose. Collapse the multi-line prose into terse one-line-per-rule legends and trailing comments; keep the load-bearing server-vs-payload distinction. No rule changes. * fix(security): address PR review feedback on S/TRY triage - Re-add `except HTTPException: raise` guard in `create_task_module` so a future dependency raising HTTPException isn't downgraded to 500 by the trailing broad catch. - Move `helpers.py` S110 noqa from file-wide per-file-ignore to inline on the three Mimikatz cred-parsing blocks — the file is server-side code and a blanket ignore would auto-excuse any future stray `except: pass`. * chore(security): polish LOW-severity PR review feedback - Add direct tests for get_random_string + generate_sessionid (mirrors TestRandomString pattern in test_helpers; previously covered only indirectly via downstream stager/agent paths). - Convert remaining log.error(..., exc_info=True) → log.exception across agent_communication_service, db/base, plugin_service for consistency with the rest of the TRY sweep. - Tighten MD5 staging-key comment in listener_service: the 32-char derived key (not the md5 hash itself) is the protocol artifact shared with deployed agents. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ation + backup_db TCP fix (#1487) * feat(alembic): add 0002 perf-indexes migration PR EmpireProject#1307 added performance indexes via SQLAlchemy model declarations (`index=True` on users.username / agent_files.session_id / agent_files.parent_id, and the composite Index on agents(listener, archived)). Fresh installs get these via Base.metadata.create_all, but existing databases do not — create_all never adds indexes to existing tables. This migration backfills the four indexes onto existing databases. Operators apply it manually with: poetry run alembic -c empire/server/core/db/alembic.ini upgrade head The migration is idempotent: upgrade() skips any index that already exists (so fresh installs no-op) and downgrade() skips any that don't. Index names match SQLAlchemy's auto-generated names so autogenerate shows no drift. A missing referenced table raises RuntimeError with an operator-friendly remediation message. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backup): force --protocol=tcp on mysqldump when port is specified When a port is present in database_config.url, backup_db() built `-h <host> -P <port>`. MySQL clients special-case `-h localhost` and connect over the Unix socket regardless of `-P`, so the dump could land on whatever MySQL is bound to the host's socket instead of the intended server. Adding `--protocol=tcp` whenever a port is specified forces the TCP connection. A comment documents the quirk so it isn't "simplified" back out. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(alembic): verify 0002 roundtrip + --protocol=tcp - test_migration_0002_upgrade_downgrade_roundtrip: stamps to head, downgrades 0002 -> 0001 and asserts the four indexes are dropped, then re-upgrades and asserts they return, then upgrades again to prove idempotency. Skipped on MySQL, where ix_agent_files_parent_id can't be dropped while the FK on parent_id exists (MySQL requires an index on FK columns) — SQLite exercises the full roundtrip. - Extend test_backup_db_mysql_port_parsing to assert `--protocol=tcp` is present when the URL carries a port. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(database): add manual upgrade command for migrations Document that Empire stamps fresh databases at baseline on startup but does not auto-apply migrations, and give the alembic upgrade command operators run after upgrading to a release with new migrations. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(alembic): keep existing migration tests green after adding 0002 Shipping 0002 moves the Alembic head from 0001 to 0002, which broke the pre-existing test_alembic.py tests in two ways: - Tests that ran migrate_db() (upgrade-to-head) then asserted the version was "0001" now land at head 0002. - The throwaway fixture migrations in test_real_migration_*, test_migrate_db_applies_pending, and test_failed_migration hardcoded revision "0002", colliding with the shipped 0002 ("Revision 0002 is present more than once"). Fixes, keeping startup's stamp-only-at-baseline semantics intact (the resting state is still 0001; only explicit migrate_db() calls reach head): - Add _get_head_revision() to base.py and a thin _head_revision() test wrapper; migrate-to-head assertions now compare against the actual head. - Re-chain the fixture migrations off the real head with unique revision ids so they no longer collide with the shipped 0002. - Add a function-scoped _restore_db_to_baseline_after fixture so the session-scoped DB's alembic_version is reset to baseline after a test mutates it (prevents version-state leak; test_startup_does_not_restamp still asserts the tracked DB stays at 0001). test_alembic.py is green on both legs: MySQL 18 passed / 4 skipped, SQLite 22 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(database): correct alembic upgrade command path The documented upgrade command pointed at `empire/server/core/db/alembic.ini`, which does not exist on this branch — that packaged ini is only added by the larger startup-wiring variant. The working config on this branch is the repo-root `alembic.ini` (with a relative `script_location`), so the command must use `-c alembic.ini` and be run from the Empire root directory. Verified: `alembic -c alembic.ini heads` returns `0002 (head)`; the previously documented path failed with "No 'script_location' key found". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(alembic): make 0002 downgrade MySQL-safe for the FK-backed index 0002.downgrade() unconditionally dropped every index, but on MySQL/InnoDB ix_agent_files_parent_id is the index backing the self-referential agent_files.parent_id -> agent_files.id foreign key. InnoDB requires an index on every FK column and refuses to drop the last one (ERROR 1553), so the downgrade failed on MySQL — the roundtrip test had to skip MySQL to stay green, masking the broken path. Skip dropping that one index on MySQL (the FK needs it regardless of this migration); SQLite has no such rule and still drops it. The roundtrip test now runs on both engines and asserts the engine-specific behavior, so MySQL exercises the full downgrade instead of skipping it. MySQL: 19 passed / 3 skipped; SQLite: 22 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(database): correct startup stamp revision (baseline 0001, not head) The Migrations section claimed startup_db() stamps the database "at the current Alembic head revision," but `_stamp_alembic_baseline()` calls `command.stamp(cfg, "0001")` — it stamps the *baseline*, not head. This also contradicted the later note (correct) that Empire "stamps fresh databases at baseline." Align the two so the docs match the code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…stubs (F022, F050) (#1492) - F022: delete __iter__/__getitem__ on ListenerUpdateRequest / StagerUpdateRequest. They reference self.__root__ (a Pydantic-v1 idiom) which is dead under Pydantic v2 — calling them raises AttributeError. Nothing iterates or indexes these models. - F050: delete test_socket_hooks.py, which held only two pass-bodied stub tests that counted as passing coverage without asserting anything. Net-negative lines; no behavior change. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EmpireProject#1356) Add a service-layer guard in update_user that blocks demoting or disabling a user when doing so would leave zero enabled admins. Catches both self-update and admin-on-admin paths, and documents the best-effort (non-row-locked) semantics of the check. Tests cover self-disable and self-demote of the last admin (blocked), non-admin disable (allowed), and demote-with-other-admins-present (allowed). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
v6.7.0 into private-main
….0 (#1539) Bumps [MishaKav/pytest-coverage-comment](https://github.com/mishakav/pytest-coverage-comment) from 1.8.0 to 1.10.0. - [Release notes](https://github.com/mishakav/pytest-coverage-comment/releases) - [Changelog](https://github.com/MishaKav/pytest-coverage-comment/blob/main/CHANGELOG.md) - [Commits](MishaKav/pytest-coverage-comment@v1.8.0...v1.10.0) --- updated-dependencies: - dependency-name: MishaKav/pytest-coverage-comment dependency-version: 1.10.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6 to 7. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v6...v7) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
v6.7.1 into private-main
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR was automatically generated by the
release-public-startworkflow.This PR should be merged with a merge commit, not a squash commit.
Merging this PR will trigger a tag and release automatically.