Skip to content

fix: API-key usage counter must not deny authentication (intermittent 403s) - #8

Merged
rutgerhofste merged 1 commit into
19.0from
fix/apikey-counter-must-not-deny-auth
Sep 7, 2026
Merged

fix: API-key usage counter must not deny authentication (intermittent 403s)#8
rutgerhofste merged 1 commit into
19.0from
fix/apikey-counter-must-not-deny-auth

Conversation

@rutgerhofste

Copy link
Copy Markdown
Member

The bug

_check_credentials bumps the usage counter with one UPDATE per authenticated call:

UPDATE res_users_apikeys
SET x_last_used = now() at time zone 'utc', x_use_count = x_use_count + 1
WHERE id = %s

When two requests using the same API key arrive concurrently, both update that row. At Odoo's REPEATABLE READ isolation the loser gets SerializationFailure (SQLSTATE 40001), which aborts the request transaction.

Odoo does know how to handle this — PG_CONCURRENCY_EXCEPTIONS_TO_RETRY in odoo/service/model.py lists SerializationFailure as retryable. But retrying() never sees it. The counter runs inside authentication, and _authenticate_explicit() in odoo/addons/base/models/ir_http.py does:

except Exception:
    _logger.info("Exception during request Authentication.", exc_info=True)
    raise AccessDenied()

So a retryable concurrency blip is converted to a hard 403 before the retry layer is reached, and a perfectly valid key is rejected. Bookkeeping takes down auth.

Seen in production

2026-07-15 08:15, key id 55, parallel crm.lead / res.partner calls from an MCP client:

ERROR odoo.sql_db: bad query: b"UPDATE res_users_apikeys SET x_last_used = ..."
ERROR: could not serialize access due to concurrent update
INFO odoo.addons.base.models.ir_http: Exception during request Authentication.
psycopg2.errors.SerializationFailure: could not serialize access due to concurrent update
... "POST /json/2/res.partner/check_access_rights HTTP/1.0" 403

Note this is intermittent and load-dependent — it only bites when the same key is used in parallel, which is exactly what a busy MCP client does.

The fix

Wrap the counter in a savepoint and swallow concurrent-update errors. The counter is best-effort; authentication is not.

Dropping an increment is free: x_use_count and x_last_used are readonly display fields, referenced only by the form view. Nothing reads them for cap enforcement or any other decision, so a lost count under contention has no downstream effect. A denied API call does.

Unexpected psycopg2.Errors are also swallowed (at warning, with traceback) rather than failing auth — same reasoning.

Verification

The load-bearing assumption is that ROLLBACK TO SAVEPOINT actually recovers a transaction after a 40001, rather than leaving it unusable. Confirmed against PostgreSQL with two concurrent sessions reproducing the exact production error:

SAVEPOINT
ERROR:  could not serialize access due to concurrent update
ROLLBACK
               verdict                | count
--------------------------------------+-------
 TRANSACTION STILL USABLE AFTER 40001 |     1
COMMIT

Also confirmed on the Odoo 19 runtime: all three psycopg2.errors classes resolve, cr.savepoint(flush=False) matches the signature in odoo/sql_db.py, and Savepoint.__exit__ rolls back but re-raises — hence the explicit try/except. ruff check and ruff format --check pass.

No unit test. tests/test_apikeys.py documents that _check_credentials needs a live odoo.http.request and is therefore covered by manual smoke plus staging rather than TransactionCase; this change doesn't alter that. Reproducing a genuine serialization failure additionally needs two concurrent transactions, which the single-cursor test harness can't express. Suggest a staging smoke: hammer one key with parallel calls and confirm no 403s.

Notes for review

🤖 Generated with Claude Code

`_check_credentials` bumps x_last_used / x_use_count with one UPDATE per
authenticated call. Two concurrent requests on the same key update that
row at once, and at REPEATABLE READ the loser gets a SerializationFailure.

Odoo's retrying() lists SerializationFailure as retryable, but never sees
this one: the counter runs inside authentication, and
_authenticate_explicit() wraps auth in `except Exception: raise
AccessDenied()`. The concurrency blip is converted to a hard 403 before
the retry layer is reached, so a perfectly valid key is rejected.

Observed on production 2026-07-15 08:15 (key id 55) — MCP clients making
parallel calls got intermittent 403s.

Run the UPDATE in a savepoint and swallow concurrency errors, so a lost
increment can't fail the request. x_use_count and x_last_used are
readonly display fields and gate nothing, so dropping one is free.

Verified against PostgreSQL that ROLLBACK TO SAVEPOINT recovers a
transaction after a 40001 and lets it commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@rutgerhofste
rutgerhofste force-pushed the fix/apikey-counter-must-not-deny-auth branch from d92c6a7 to 3a54560 Compare September 7, 2026 17:22
@rutgerhofste

Copy link
Copy Markdown
Member Author

Rebased on 19.0 (now at d3f8d2c, after #9). Only the manifest version conflicted; the code applied clean. Version resolved to 19.0.1.20.4 on top of 19.0.1.20.3.

The bug is still live: production hit it again today, 2026-09-07 17:01 UTC, five SerializationFailures in 200ms on key id 33.

ERROR odoo.sql_db: bad query: b"UPDATE res_users_apikeys SET x_last_used = now() at time zone 'utc', x_use_count = x_use_count + 1 WHERE id = 33"
ERROR: could not serialize access due to concurrent update

Different key from the July report (33, was 55), same failure mode.

🤖 Generated with Claude Code

@rutgerhofste
rutgerhofste merged commit cb608a4 into 19.0 Sep 7, 2026
3 checks passed
@rutgerhofste
rutgerhofste deleted the fix/apikey-counter-must-not-deny-auth branch September 7, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant