diff --git a/CHANGELOG.md b/CHANGELOG.md index a8869b4..95fdce4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,14 @@ All notable changes to this project are documented here. ## [Unreleased] +## [0.3.3] - 2026-07-30 + +- docs: state plainly what the default `SensitiveDataFilter` does. It matches keywords; it is not a secret-detection engine, and the Quickstart no longer implies otherwise +- docs: add a `Limitations` section naming what the default filter does and does not catch, and a separate `Roadmap` section for the hosted dashboard +- docs: note that `@shield_tool` handles sync and async functions and logs a structured warning to stderr on a block +- docs: record `file://`-style disallowed schemes among the Default Deny cases +- docs: correct the 0.2.x changelog attributions, which were one release out of step with the tags + ## [0.3.2] - 2026-07-30 - fix: `@shield_tool` now wraps coroutine functions and async generators in kind. Previously it always produced a sync wrapper, so `inspect.iscoroutinefunction()` returned `False` on a shielded `async def` and frameworks (LangChain, MCP) never awaited it — the model received a coroutine `repr` instead of the tool's result @@ -25,23 +33,26 @@ All notable changes to this project are documented here. - feat: `modelfuzz scan` is now an adaptive fuzzer — it evolves refused attacks into more deceptive variants and retries within a `--budget-s` time budget, instead of sending a fixed list of static prompts - test: add coverage for the scan mutation loop, budget handling, and error paths - ci: run tests with all extras installed so the `scan` path is exercised + +## [0.2.2] - 2026-07-23 + - fix: point PyPI `Homepage` at https://www.modelfuzz.com instead of the GitHub repo - docs: add a zero-clone "Try It Now" snippet, verified against the published PyPI package - docs: add CHANGELOG.md and SECURITY.md - docs: link website, LinkedIn, and PyPI from the README - docs: show real `demo.py` and `modelfuzz scan` output with runnable commands - -## [0.2.1] - 2026-07-22 - - docs: clarify default `PolicyEngine` behavior in the Quickstart - docs: document the `modelfuzz scan` CLI - docs: add Scarf analytics pixel to README +## [0.2.1] - 2026-07-22 + +- feat: add confused-deputy and authority-escalation scan payloads + ## [0.2.0] - 2026-07-22 - feat: add the offensive scanner (`modelfuzz scan`) — red-teams an OpenAI-compatible endpoint with prompt-injection payloads and reports which ones trick it into an unsafe tool call - feat: report `⚠️ INCONCLUSIVE` instead of a false-safe result when every prompt errors out -- feat: add confused-deputy and authority-escalation scan payloads ## [0.1.3] - 2026-07-21 @@ -58,7 +69,12 @@ All notable changes to this project are documented here. - CI workflow (lint + tests), MIT license, unit/integration test suite - Package renamed from `agentshield` to `modelfuzz` -[Unreleased]: https://github.com/higagan/modelfuzz/compare/v0.2.1...HEAD +[Unreleased]: https://github.com/higagan/modelfuzz/compare/v0.3.3...HEAD +[0.3.3]: https://github.com/higagan/modelfuzz/compare/v0.3.2...v0.3.3 +[0.3.2]: https://github.com/higagan/modelfuzz/compare/v0.3.1...v0.3.2 +[0.3.1]: https://github.com/higagan/modelfuzz/compare/v0.3.0...v0.3.1 +[0.3.0]: https://github.com/higagan/modelfuzz/compare/v0.2.2...v0.3.0 +[0.2.2]: https://github.com/higagan/modelfuzz/compare/v0.2.1...v0.2.2 [0.2.1]: https://github.com/higagan/modelfuzz/compare/v0.2.0...v0.2.1 [0.2.0]: https://github.com/higagan/modelfuzz/compare/v0.1.3...v0.2.0 [0.1.3]: https://github.com/higagan/modelfuzz/compare/v0.1.2...v0.1.3 diff --git a/README.md b/README.md index b93f16e..352d0d5 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ def send_email(to_address: str, subject: str, body: str) -> None: smtp.send(to_address, subject, body) ``` -> **Note:** When used bare, `@shield_tool` applies a default `PolicyEngine` that blocks common secrets (API keys, passwords). To define custom rules (like `URLAllowList`), simply pass your own engine: `@shield_tool(engine=my_engine)`. +> **Note:** When used bare, `@shield_tool` applies a default `PolicyEngine` with a basic `SensitiveDataFilter`. For production use, define your own rules (like `URLAllowList` or custom secret scanners) and pass your own engine: `@shield_tool(engine=my_engine)`. Works bare (`@shield_tool`) or called (`@shield_tool()`) — both wrap `send_email` identically. Any argument that trips a policy raises `ModelFuzzBlockError` before the function body runs. @@ -99,8 +99,16 @@ Output: ## How It Works - **`PolicyEngine`** — runs an ordered list of policies against every tool-call argument and short-circuits on the first violation. Policies are plain callables (`(value) -> Violation | None`), so writing your own is a one-function job. -- **`@shield_tool` decorator** — wraps any function so every positional and keyword argument passes through the engine before the function body runs. A violation raises `ModelFuzzBlockError`; the tool never executes. -- **Default Deny** — allowlist rules like `URLAllowList` block anything not explicitly permitted: unknown domains, userinfo tricks (`http://api.internal.com@evil.com`), and unparseable URLs are all treated as violations. When in doubt, the call doesn't run. +- **`@shield_tool` decorator** — wraps any function (sync or async) so every positional and keyword argument passes through the engine before the function body runs. A violation raises `ModelFuzzBlockError` and logs a structured warning to stderr; the tool never executes. +- **Default Deny** — allowlist rules like `URLAllowList` block anything not explicitly permitted: unknown domains, userinfo tricks (`http://api.internal.com@evil.com`), disallowed schemes (`file://`), and unparseable URLs are all treated as violations. When in doubt, the call doesn't run. + +## Limitations + +ModelFuzz provides the interception point, the policy protocol, and an adaptive fuzzer. The default `SensitiveDataFilter` matches the literal strings `secret`, `password`, and `api_key` — it does not recognise credential formats, so a real `sk-…` or `AKIA…` key will pass through it. Treat it as a demo default and write policies for your own threat model. Also: policies see each argument in isolation, not the whole call, and only `str`, `list`, `tuple`, and `dict` values are inspected. + +## Roadmap + +A hosted dashboard is in development, providing centralized audit logs, policy versioning, and managed secret detection. ## Red-Team Scanner diff --git a/pyproject.toml b/pyproject.toml index c32c31f..8c12d5d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "modelfuzz" -version = "0.3.2" +version = "0.3.3" description = "Runtime guardrails for AI agents." readme = "README.md" license = { text = "MIT" } diff --git a/uv.lock b/uv.lock index 01b8939..99b4a4f 100644 --- a/uv.lock +++ b/uv.lock @@ -286,7 +286,7 @@ wheels = [ [[package]] name = "modelfuzz" -version = "0.3.2" +version = "0.3.3" source = { editable = "." } dependencies = [ { name = "typer" },