Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 21 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@ All notable changes to this project are documented here.

## [Unreleased]

## [0.3.3] - 2026-07-30

- docs: state plainly what the default `SensitiveDataFilter` does. It matches keywords; it is not a secret-detection engine, and the Quickstart no longer implies otherwise
- docs: add a `Limitations` section naming what the default filter does and does not catch, and a separate `Roadmap` section for the hosted dashboard
- docs: note that `@shield_tool` handles sync and async functions and logs a structured warning to stderr on a block
- docs: record `file://`-style disallowed schemes among the Default Deny cases
- docs: correct the 0.2.x changelog attributions, which were one release out of step with the tags

## [0.3.2] - 2026-07-30

- fix: `@shield_tool` now wraps coroutine functions and async generators in kind. Previously it always produced a sync wrapper, so `inspect.iscoroutinefunction()` returned `False` on a shielded `async def` and frameworks (LangChain, MCP) never awaited it — the model received a coroutine `repr` instead of the tool's result
Expand All @@ -25,23 +33,26 @@ All notable changes to this project are documented here.
- feat: `modelfuzz scan` is now an adaptive fuzzer — it evolves refused attacks into more deceptive variants and retries within a `--budget-s` time budget, instead of sending a fixed list of static prompts
- test: add coverage for the scan mutation loop, budget handling, and error paths
- ci: run tests with all extras installed so the `scan` path is exercised

## [0.2.2] - 2026-07-23

- fix: point PyPI `Homepage` at https://www.modelfuzz.com instead of the GitHub repo
- docs: add a zero-clone "Try It Now" snippet, verified against the published PyPI package
- docs: add CHANGELOG.md and SECURITY.md
- docs: link website, LinkedIn, and PyPI from the README
- docs: show real `demo.py` and `modelfuzz scan` output with runnable commands

## [0.2.1] - 2026-07-22

- docs: clarify default `PolicyEngine` behavior in the Quickstart
- docs: document the `modelfuzz scan` CLI
- docs: add Scarf analytics pixel to README

## [0.2.1] - 2026-07-22

- feat: add confused-deputy and authority-escalation scan payloads

## [0.2.0] - 2026-07-22

- feat: add the offensive scanner (`modelfuzz scan`) — red-teams an OpenAI-compatible endpoint with prompt-injection payloads and reports which ones trick it into an unsafe tool call
- feat: report `⚠️ INCONCLUSIVE` instead of a false-safe result when every prompt errors out
- feat: add confused-deputy and authority-escalation scan payloads

## [0.1.3] - 2026-07-21

Expand All @@ -58,7 +69,12 @@ All notable changes to this project are documented here.
- CI workflow (lint + tests), MIT license, unit/integration test suite
- Package renamed from `agentshield` to `modelfuzz`

[Unreleased]: https://github.com/higagan/modelfuzz/compare/v0.2.1...HEAD
[Unreleased]: https://github.com/higagan/modelfuzz/compare/v0.3.3...HEAD
[0.3.3]: https://github.com/higagan/modelfuzz/compare/v0.3.2...v0.3.3
[0.3.2]: https://github.com/higagan/modelfuzz/compare/v0.3.1...v0.3.2
[0.3.1]: https://github.com/higagan/modelfuzz/compare/v0.3.0...v0.3.1
[0.3.0]: https://github.com/higagan/modelfuzz/compare/v0.2.2...v0.3.0
[0.2.2]: https://github.com/higagan/modelfuzz/compare/v0.2.1...v0.2.2
[0.2.1]: https://github.com/higagan/modelfuzz/compare/v0.2.0...v0.2.1
[0.2.0]: https://github.com/higagan/modelfuzz/compare/v0.1.3...v0.2.0
[0.1.3]: https://github.com/higagan/modelfuzz/compare/v0.1.2...v0.1.3
Expand Down
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ def send_email(to_address: str, subject: str, body: str) -> None:
smtp.send(to_address, subject, body)
```

> **Note:** When used bare, `@shield_tool` applies a default `PolicyEngine` that blocks common secrets (API keys, passwords). To define custom rules (like `URLAllowList`), simply pass your own engine: `@shield_tool(engine=my_engine)`.
> **Note:** When used bare, `@shield_tool` applies a default `PolicyEngine` with a basic `SensitiveDataFilter`. For production use, define your own rules (like `URLAllowList` or custom secret scanners) and pass your own engine: `@shield_tool(engine=my_engine)`.

Works bare (`@shield_tool`) or called (`@shield_tool()`) — both wrap `send_email` identically. Any argument that trips a policy raises `ModelFuzzBlockError` before the function body runs.

Expand Down Expand Up @@ -99,8 +99,16 @@ Output:
## How It Works

- **`PolicyEngine`** — runs an ordered list of policies against every tool-call argument and short-circuits on the first violation. Policies are plain callables (`(value) -> Violation | None`), so writing your own is a one-function job.
- **`@shield_tool` decorator** — wraps any function so every positional and keyword argument passes through the engine before the function body runs. A violation raises `ModelFuzzBlockError`; the tool never executes.
- **Default Deny** — allowlist rules like `URLAllowList` block anything not explicitly permitted: unknown domains, userinfo tricks (`http://api.internal.com@evil.com`), and unparseable URLs are all treated as violations. When in doubt, the call doesn't run.
- **`@shield_tool` decorator** — wraps any function (sync or async) so every positional and keyword argument passes through the engine before the function body runs. A violation raises `ModelFuzzBlockError` and logs a structured warning to stderr; the tool never executes.
- **Default Deny** — allowlist rules like `URLAllowList` block anything not explicitly permitted: unknown domains, userinfo tricks (`http://api.internal.com@evil.com`), disallowed schemes (`file://`), and unparseable URLs are all treated as violations. When in doubt, the call doesn't run.

## Limitations

ModelFuzz provides the interception point, the policy protocol, and an adaptive fuzzer. The default `SensitiveDataFilter` matches the literal strings `secret`, `password`, and `api_key` — it does not recognise credential formats, so a real `sk-…` or `AKIA…` key will pass through it. Treat it as a demo default and write policies for your own threat model. Also: policies see each argument in isolation, not the whole call, and only `str`, `list`, `tuple`, and `dict` values are inspected.

## Roadmap

A hosted dashboard is in development, providing centralized audit logs, policy versioning, and managed secret detection.

## Red-Team Scanner

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "modelfuzz"
version = "0.3.2"
version = "0.3.3"
description = "Runtime guardrails for AI agents."
readme = "README.md"
license = { text = "MIT" }
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading