feat: route long single-field payloads through the local filesystem - #55
feat: route long single-field payloads through the local filesystem#55ERnsTL wants to merge 10 commits into
Conversation
…v1.3.0)
LLMs routinely mistransform long HTML / source code / rich text that
round-trips through the JSON-RPC envelope (double-escaped quotes,
escaped newlines, embedded Unicode). Two new MCP tools let agents
move those payloads out of band: only the SHA-256 + size fingerprint
crosses the wire, the real bytes stay on disk. Destructive write
goes through the same preview/execute + confirm gate as every other
write.
- read_field_to_file (read domain): stream one field's value to a
local file. Honors field ACL (redacted values become
[REDACTED by field ACL] placeholder, response flags it via
field_was_redacted). Text defaults to UTF-8; binary defaults to
base64. Symlink-safe, refuses overwrite, absolute-path-only,
O_NOFOLLOW on the create fd.
- write_field_from_file (write domain): two-phase preview/execute
flow mirroring chatter_post. Preview returns a token carrying
only sha256:<hex>:<size>; execute re-reads the file, re-checks
the hash (catches tamper + TOCTOU), fetches live fields_get
metadata (refuses readonly=True), then routes through
confirm=true + ODOO_MCP_ENABLE_WRITES=1. Binary round-trip
via base64; otherwise UTF-8.
- ODOO_MCP_FIELD_FILE_ROOTS allow-list: os.pathsep-separated
absolute directories. Hardened pattern from
ODOO_MCP_ATTACHMENT_UPLOAD_ROOTS: Path.resolve()d before the
containment check, so .. traversal and symlink escapes cannot
reach outside the operator's allow-list. Per-call override via
the file_root argument on both tools.
- Fail-closed default with actionable error: with no env var and
no per-call override, both tools reject every call. The error
names the env var, lists platform-specific safe defaults
(~/.cache/odoo-mcp/field-files on Linux/macOS,
%LOCALAPPDATA%\odoo-mcp\Cache\field-files on Windows), and
explicitly warns against /tmp (typically world-readable — long
field payloads would leak to other local users / processes).
- Runtime posture in health_check.runtime.field_file_roots
(env var name + count + resolved paths, no secrets).
- docs/field-file-io.md: exhaustive operator guide (config,
examples for HTML comments, binary attachments, long internal
notes; full threat-model table).
- CHANGELOG 1.3.0 entry. pyproject version 1.2.1 -> 1.3.0.
Fixed: write_field_from_file execute_kw argument shape. The initial
implementation called
odoo.execute_method(model, "write", [[ids], {vals}])
which OdooClient.execute_method(self, model, method, *args, **kwargs)
forwarded as a single positional arg, so Odoo's XML-RPC dispatcher
saw execute_kw("project.task", "write", [[ids], {vals}]) and unpacked
it as ProjectTask.write(*[[ids], {vals}]) -> write([ids], {vals}),
faulting with `missing 1 required positional argument: 'vals'`.
Now called as `odoo.execute_method(model, "write", [ids], {vals})`
so ids + vals arrive as two separate positional arguments (mirrors
execute_approved_write). New regression test
test_write_field_from_file_does_not_pack_args_into_single_list; the
execute_method test stub now records the unpacked *args form,
matching OdooClient's signature, so a re-introduction of the
nested-list form fires immediately.
932 tests pass (was 909). Tool count 41 -> 43 (read_field_to_file,
write_field_from_file). No behavior change for existing tools.
- Wrap all Optional[List[...]], List[...], and Optional[Dict[...]] parameters in Annotated[..., Field(description=...)] so Pydantic emits a clean JSON array/object schema instead of the {"item": [...]} wrapper that caused MCP clients to reject calls with `Input should be a valid list [type=list_type, input_value={'item': [...]}, input_type=dict]`.
- Fix `get_model_fields`, `search_records`, `read_record`, `aggregate_records` in `src/odoo_mcp/tools_read.py`.
- Fix `index_knowledge` in `src/odoo_mcp/tools_knowledge.py`.
- Fix `data_quality_report` in `src/odoo_mcp/tools_data_quality.py`.
- Fix `generate_json2_payload`, `inspect_model_relationships`, `analyze_upgrade_log`, `fit_gap_report`, `scan_addons_source`, `build_domain` in `src/odoo_mcp/tools_diagnostics.py`.
- Fix `preview_write`, `validate_write`, `chatter_post` in `src/odoo_mcp/tools_write.py`.
- Fix `search_across_instances`, `aggregate_across_instances` in `src/odoo_mcp/tools_cross_instance.py`.
- Bump version 1.3.0 -> 1.3.1 in `pyproject.toml` (patch release, no behaviour change at runtime).
- Add `[1.3.1] - 2026-07-21` entry to `CHANGELOG.md` documenting the JSON-schema wrapping fix.
- Verified end-to-end: 932/932 pytest tests pass with the new `mcp>=1.27,<2` (installed 1.28.1); direct `inputSchema` inspection of the running server confirms list/dict parameters now export as proper JSON arrays and objects.
|
@tuanle96 Thanks for the feedback - I will resolve these points you have mentioned. |
- file_root: treat as selector among configured roots (cannot widen allow-list) - file_root: always require ODOO_MCP_FIELD_FILE_ROOTS; no free-form escape hatch - file_root: check absolute-ness before resolve() (kills $CWD/scratch bypass) - restrict_field_file_path: validate against all roots when no override - restrict_field_file_path: fix docstring (points at correct functions) - read_field_to_file: reject encoding="base64" on non-binary fields - read_field_to_file: switch base64 decode to validate=True - write_field_from_file: rewrite field/encoding descriptions to match behavior - tests: add 5 regression tests + update 2 existing tests - docs: update field-file-io.md (selector semantics, no file_root remediation)
|
Thanks @tuanle96 for the thorough review - all 6 comments addressed. Summary below; full details in the commit. 🔒 Security fix —
Fix:
🐛 Validation gap - The documented contract said "cannot force base64 on a non-binary type" but the code only did Fix:
📝 Docs gap - The description said "set Fix: rewrote the 🧪 Tests
📚 Docs
Verification:
|
|
@tuanle96 New commit fixing validation errors when using certain tools. Applied the solution already used in many other tools. SummaryFixes Pydantic schema generation for list/dict tool parameters across 19 MCP tools. Parameters declared as WhyBefore the fix, agents could not pass array arguments to several read, write, and diagnostics tools - every call returned a Changes
No behavior change at runtime - only the JSON-Schema surface for Environment / Dependency NoteThe fix relies on the Verification
|
|
@tuanle96 security comments fixed, fixed several tools (datastructure wrapping issue) and resolved merge conflicts. |
|
New fix release. SummaryPatch release What changed
Files touched11 files, 401 +/36 -. Compatibility
Test evidenceEnd-to-end smoke through the deployed Checklist
|
…omain parsing
The four most common agent-side tool errors — ``measures=""`` (Pydantic
type mismatch), ``measures=["__count"]`` (Odoo rejects the auto-count
column as a measure), ``domain="def x():"`` (malformed domain) and
``instance="ghost"`` (unknown instance) — used to leak raw Pydantic
stack traces or hit Odoo's error path before any agent-friendly envelope
ran. This change delivers the friendly ``{"success": false, "tool": ...,
"error": "Invalid input: ..."}`` envelope on every one of them, removes
a CPython runpy RuntimeWarning by hand, and bumps 1.3.1 → 1.3.2 per
semver. No public-API breakage; 944 tests pass; 0 warnings.
What changed
- Friendly envelope on Pydantic argument validation. FastMCP validates
annotated tool parameters via Pydantic BEFORE the tool body runs
(``mcp.server.fastmcp.utilities.func_metadata.call_fn_with_arg_validation``),
and converts any ``ValidationError`` to ``mcp.server.fastmcp.exceptions.ToolError``
whose ``__cause__`` is the original error. The bound method
``FastMCP.call_tool`` is captured by reference at instance init via
``self._mcp_server.call_tool(validate_input=False)(self.call_tool)``, and
any post-init monkey-patch on ``mcp.call_tool`` or ``mcp._tool_manager.call_tool``
bypasses the JSON-RPC dispatch path. The fix is a tiny
``_TranslationAwareFastMCP(FastMCP)`` subclass in ``src/odoo_mcp/server_core.py``
that overrides ``call_tool``; FastMCP's init captures the bound subclass
method, so the registered JSON-RPC handler IS the translator. It catches
``BaseException``, narrows to ``ToolError`` whose ``__cause__`` is a
``pydantic.ValidationError``, then re-emits a friendly envelope — a dict
shaped ``{success, tool, error}`` for tools with ``structured_output=True``
(satisfied by ``aggregate_records``, ``search_records``, …) and a
``[TextContent(...)]`` content-block list otherwise. Runtime errors,
Odoo faults, and non-validation tool exceptions pass through untouched.
- ``normalize_domain_input`` now raises ``ValueError`` on garbage input.
Previously it silently returned ``[]`` and callers queried the wrong
record set with no error. The friendly message names both legal forms:
a JSON list ``[["is_timesheet", "=", true], ["project_id", "=", 37]]``
and the ``&``-prefixed Python literal that ``ast.literal_eval`` accepts.
``tests/test_tool_helpers.py`` gained three raise-path tests; two
pre-existing tests in ``tests/test_server.py`` were renamed
(``..._returns_empty_...`` → ``..._raises_...``) to reflect the new contract.
- Tightened ``aggregate_records`` description with a ``__count`` warning.
``__count`` is the row-count pseudo-column that Odoo's ``read_group`` /
``formatted_read_group`` returns on every row automatically; passing it
explicitly as a measure produces ``Invalid field '__count'``. The
description spells this out so agents stop guessing. Same tool gains
a ``list_instances`` hint that the other tools already carry.
- ``RuntimeWarning`` removed from the CLI test suite (root-cause fix).
``runpy.run_module("odoo_mcp.__main__", run_name="__main__")`` triggered
CPython's standard ``RuntimeWarning: 'odoo_mcp.__main__' found in
sys.modules after import of package 'odoo_mcp', but prior to execution``
whenever the test suite had already imported the package. Replaced with
a direct invocation of the entry-point expression ``sys.exit(cli.main())``
— same contract tested, no ``sys.modules`` pollution, no warning. No
``pytest.ini`` filter, no ``# noqa`` markers.
Dead-code cleanup
The first implementation tried ``@safe_tool_call`` decorators on the
function bodies. Verification proved that path can never reach a
``ValidationError`` (FastMCP validates parameters before calling the
function), so the decorator was always a no-op. Removed:
- ``safe_tool_call`` itself, plus its ``functools.wraps`` boilerplate and
five tests that exercised it. ``src/odoo_mcp/error_handling.py`` is
now reduced to its single reusable helper, ``_format_validation_error``
(bounded single-line renderer of ``pydantic.ValidationError``).
- Two ``@safe_tool_call`` decorators that were applied to
``aggregate_records`` and ``search_records`` in ``tools_read.py``.
- ``from .error_handling import safe_tool_call`` import (comment now
documents the move to the subclass layer).
Files touched
CHANGELOG.md | 35 ++ (1.3.2 entry)
pyproject.toml | 2 +- (1.3.1 -> 1.3.2)
server.json | 4 +- (1.3.1 -> 1.3.2)
src/odoo_mcp/error_handling.py | 63 ++ (NEW, then trimmed to helper only)
src/odoo_mcp/server_core.py | 78 ++ (NEW subclass _TranslationAwareFastMCP)
src/odoo_mcp/tool_helpers.py | 19 +- (normalize_domain_input raises)
src/odoo_mcp/tools_read.py | 25 ++ (description tweaks, decorator removed)
tests/test_cli.py | 33 +- (no more runpy)
tests/test_error_handling.py | 60 ++ (NEW translator + helper tests)
tests/test_server.py | 16 +- (2 tests renamed for raise contract)
tests/test_tool_helpers.py | 45 +- (raise-path coverage)
Public-API compatibility
- Same 41 tools, same parameter sets, same envelopes (the envelope the
failure path now produces is byte-identical to what a well-formed call
would produce — only the ``success`` flag differs).
- ``measure`` argument is unchanged; only its ``description`` string grew
by one warning. String change only, callers that pin ``"sum"|"avg"|...
via explicit ``measures=["amount_total:sum"]`` are unaffected.
- No new environment variables, no new dependencies, no new
configuration knobs.
Test evidence
$ python3 -m pytest -q
944 passed in 3.95s
$ uvx --from /home/ernst/code/mcp/mcp-odoo python - <<'PY' 2>&1 | tail
... real ToolError-with-ValidationError cause raises through the
... registered JSON-RPC dispatcher ...
type: dict
success: False
tool: 'aggregate_records'
error: 'Invalid input: measures: Value error, Input should be a valid list'
PY
Live MCP server (post-restart) confirms all four failure modes:
aggregate_records(measures="")
-> {"success": false, "tool": "aggregate_records",
"error": "Invalid input: measures: Input should be a valid list"}
search_records(domain="def x():")
-> {"success": false, "error": "Domain string is neither valid JSON
nor a Python literal: invalid syntax (...). Pass a list of
3-tuples, e.g. [\"is_timesheet\", \"=\", true], [\"project_id\",
\"=\", 37], or use the & prefix operator: ..."}
And valid input is unchanged:
aggregate_records(model="sale.order", group_by=["state"])
-> 4 rows: cancel=79, draft=17, sale=868, sent=64 (1028 orders)
Checklist
- [x] Version bumped (1.3.1 -> 1.3.2) per semver PATCH
- [x] server.json, pyproject.toml, CHANGELOG entry agree on 1.3.2
- [x] All 944 tests pass, no warnings
- [x] No public-API signature changes
- [x] No @safe_tool_call dead code, no warning filters, no # noqa
- [x] Live MCP server restart confirmed envelope shape end-to-end
…en check (v1.3.3) Regression: execute_approved_write rejected valid approval tokens with 'approval token does not match the canonical payload; re-run preview_write and validate_write' when caller-side record_ids arrived nested (e.g. [[3598, 3594, ...]]) instead of flat ([3598, 3594, ...]). Root cause: build_write_preview_report normalizes record_ids to a flat list[int] before hashing (agent_tools.py:154). verify_write_approval and write_approval_payload passed record_ids straight through from the approval dict, so any wrapper-induced nesting flipped the SHA-256 vs. the stored token. Same class of bug as the 1.2.1 _normalize_numbers int/float drift fix. Fix: - New _normalized_record_ids helper in agent_tools.py that flattens any nesting, casts digit-strings to int, drops non-digit entries (rather than coercing them to 0 -> Odoo super-user), and explicitly skips booleans (bool is an int subclass in Python). - verify_write_approval (agent_tools.py) and write_approval_payload (server_core.py) now route record_ids through the helper, matching the preview-side normalization. - 6 new regression tests in tests/test_agent_tools.py. No public API change. All 108 tests in test_agent_tools.py pass.
…envelope (v1.3.4) Regression: execute_approved_write error envelopes (token mismatch, validation record missing, payload mismatch, missing confirm, writes disabled, generic except) lacked the result key, which FastMCP's inferred outputSchema for tools with structured_output=True had marked as required from the success-path sample. The transport then rejected the envelope with 'Output validation error: result is a required property'. Fix: - New _normalize_write_response helper in tools_write.py that injects result: None on envelopes missing the key, no-op when present. - _execute_approved_write_gated routes all 7 return paths through it (5 early-return error gates + except arm + success path). - 4 new regression tests in tests/test_batch_write.py. No public API change. All 954 tests pass.
|
@tuanle96 two additional fixes fix(write-approval): normalize record_ids + inject result:None on every execute_approved_write envelope (v1.3.3 + v1.3.4)DescriptionUser-facing behavior change
Bug 1 —
|
|
@tuanle96 ready to be merged, all merge conflicts fixed |
1 participant
v1.3.0 — Field-File I/O + Cache-Bug-Fix
TL;DR
Two new MCP tools (
read_field_to_file,write_field_from_file) route long single-field payloads between Odoo and the local filesystem so LLMs no longer have to round-trip long HTML/source/rich-text content through the JSON-RPC envelope. Tool count 41 → 43. 932 tests pass.Why this release
LLMs routinely mistransform long HTML / source code / rich text that travels through the JSON-RPC envelope (double-escaped quotes, escaped newlines, embedded Unicode). Beyond the correctness pain, an 80 KB e-mail body, 50 KB product description or an Odoo Knowledge article (possibly even with embedded images) doubles in size when wrapped in JSON and lands in every subsequent tool call's history — a real context-window blow-up.
Field-file I/O solves both: the agent reads the field straight to disk, edits it locally, and writes it back. Only the SHA-256 + size fingerprint crosses the wire.
What's new
read_field_to_file(read domain) — stream one field's value to a local file. Text defaults to UTF-8; binary defaults to base64. Honors the field ACL (redacted values become[REDACTED by field ACL]placeholder, response flags it viafield_was_redacted). Symlink-safe, refuses overwrite, absolute-path-only,O_NOFOLLOWon the create fd.write_field_from_file(write domain) — two-phase preview/execute flow mirroringchatter_post. Preview returns a token carrying onlysha256:<hex>:<size>; execute re-reads the file, re-checks the hash (catches tamper + TOCTOU), fetches livefields_getmetadata (refusesreadonly=True), then routes throughconfirm=true+ODOO_MCP_ENABLE_WRITES=1. Binary round-trip via base64; otherwise UTF-8.ODOO_MCP_FIELD_FILE_ROOTSallow-list —os.pathsep-separated absolute directories. Hardened pattern fromODOO_MCP_ATTACHMENT_UPLOAD_ROOTS:Path.resolve()d before the containment check, so..traversal and symlink escapes cannot reach outside the operator's allow-list. Per-call override via thefile_rootargument on both tools.Fail-closed default with actionable error — with no env var and no per-call override, both tools reject every call. The error names the env var, lists platform-specific safe defaults (
~/.cache/odoo-mcp/field-fileson Linux/macOS,%LOCALAPPDATA%\odoo-mcp\Cache\field-fileson Windows), and explicitly warns against/tmp(typically world-readable — long field payloads would leak to other local users / processes).Threat model (summary)
/etcread_field_to_filerefuses overwrite (O_CREAT | O_EXCL);write_field_from_fileonly writes to OdooO_NOFOLLOWon create fd;Path.resolvecollapses symlinks before the containment checkO_RDONLY | O_NOFOLLOWon input fd; size cap and SHA-256 derived from that same fdread_field_to_file; redacted fields write a placeholdercontent_sha256before doing anythingBug fix (caught during operator testing)
write_field_from_fileexecute_kwargument shape. The initial implementation calledodoo.execute_method(model, "write", [[ids], {vals}]).OdooClientsplits*args, so Odoo receivedexecute_kw("project.task", "write", [[ids], {vals}])— a single positional arg. Odoo's XML-RPC dispatcher then unpacked it asProjectTask.write(*[[ids], {vals}])→write([ids], {vals}), and Odoo faulted withTypeError: ProjectTask.write() missing 1 required positional argument: 'vals'. Now called asodoo.execute_method(model, "write", [ids], {vals})so ids + vals arrive as two separate positional arguments (mirrorsexecute_approved_write, which always splatted*args).A regression test (
test_write_field_from_file_does_not_pack_args_into_single_list) plus an*args-awareexecute_methodtest stub (now matchingOdooClient's signature exactly) ensure a re-introduction of the nested-list form fires immediately. Real-world reproducer:project.taskdescriptionfield, record 134.Installation / upgrade
No migration needed. Pull, restart the MCP server.
For development workflows using
uvx --from <directory>(e.g. Cline), note that uv caches the source tree as an immutable archive in~/.cache/uv/archive-v0/. After pulling this release, runuv cache clean odoo-mcponce and restart the MCP server, or add--refreshto youruvxargs so uv auto-invalidates on every start (negligible overhead — the first build is ~3 sec and only runs when the source actually changed).Files changed
15 files, +1946 / -13.
CONTRIBUTING Checklist
pytest)ruff check— 0 issuesmypy— 0 issues across 34 source fileslint-imports— 2/2 contracts keptscripts/odoo_compose_smoke.py) — tool count 43 ✓Added+Security+Fixedsectionsdocs/field-file-io.mdoperator guidepyproject.tomlversion 1.2.1 → 1.3.0Related
project.task.description)ODOO_MCP_ATTACHMENT_UPLOAD_ROOTS(1.2.0) — same root-pattern, same fail-closed posture, same Field ACL integration