Round 17: harden legal retrieval and document admission for 0.9.2

CHANGELOG.md

@@ -7,34 +7,62 @@ are historical; consult the [current guides](docs/README.md) for present behavio
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.9.2] - 2026-11-06
 
-### Changed
+This release hardens the Turkish legal extraction and retrieval features from
+Round 16 and Round 17. V4 HTTP routes remain compatible. Production qualification
+remains **NO-GO** pending the normal certification and production-like release gates.
+
+### Added
 
-- Queries matching a decision's own docket/decision reference promote its
-  first chunk through the existing exact-match slot, shared with articles
-  (at most three chunks total). Only stored references beginning in the first
-  300 characters of an authorized first chunk count; later quotations do not.
+- Rule-based Turkish legal item extraction for newly admitted chunks, capturing
+  versioned legal rule items with chunk offsets (citation, decision_ref,
+  date, amount, duration, and outcome). Extracted items are returned in
+  search provenance under `legal_items` separately from SPO facts, without
+  affecting fusion scores. Document purge erases their source quotes.
+- Store-learned law names: authorized document titles and law headers provide
+  scope-cached and indexed law names, including the `K.` abbreviation.
+- Exact-match-first slot: queries matching a decision's own docket/decision reference
+  or exact law/article occupy at most three leading search positions in document order,
+  preserving remaining result order. Decision matching requires references beginning in
+  the first 300 characters of an authorized first chunk; later quotations do not count.
 
-- Newly admitted chunks keep versioned Turkish legal rule items with chunk
-  offsets, returned in search provenance separately from SPO facts. Item storage
-  preserves fusion scores, and document purge erases their source quotes. The existing
-  `MESA_LEGAL_RULES_ENABLED` switch controls admission.
+### Changed
 
+- `MESA_LEGAL_RULES_ENABLED` defaults to `true`. When set to `false`, it disables
+  automatic legal identity assignment and item extraction during ingestion. During
+  search, it disables legal processing (citation resolution, exact-match slot,
+  decision künye promotion, legal ranking factors, and graph citation parsing) and
+  suppresses stored legal result overlays in returned copies while keeping durable
+  identities and snapshots intact.
+- Open-ended SPO fact extraction (`MESA_EXTRACTION_ENABLED`) defaults to `false`
+  regardless of `MESA_MODEL_ENABLED`; an explicit `true` still enables it.
 - Constitutional Court names (`Anayasa Mahkemesi`, its suffix forms and `AYM`)
-  no longer activate Constitution citation ranking. Separate statute citations
-  in the same query are preserved.
-
-- Exact law/article matches now occupy at most three leading search positions
-  in document order, preserving the remaining order. Authorized document titles
-  and law headers provide scope-cached law names, including the `K.` abbreviation.
-- Document article plans assign the existing typed legal identity to ordinary
-  articles and their continuation chunks. Generic 3–5 digit law-number
-  citations use the existing retrieval lane; `MESA_LEGAL_RULES_ENABLED=false`
-  disables automatic identity assignment for new chunks. Document fallback
-  assertions preserve their existing document/chunk graph and lexical shape.
-- Open-ended SPO fact extraction (`MESA_EXTRACTION_ENABLED`) defaults to false
-  regardless of `MESA_MODEL_ENABLED`; an explicit true still enables it (D46).
+  no longer activate Constitution citation ranking; separate statute citations in
+  the same query are preserved.
+- Large-document admission waits for queue room on `queue_over_capacity` without
+  repeating full document chunk planning.
+
+### Security
+
+- Fixed synchronous CPU-bound request-path stalls that also affect 0.9.1: rule
+  extraction, citation resolution, scope building, and heavy token scoring now bound
+  caller-text work and yield thread execution, keeping `/health` responsive under
+  high load. Users of 0.9.1 are recommended to upgrade.
+
+### Known issues
+
+- Heavy 1,000-candidate search takes approximately 11.6 s; `/health` responsiveness
+  is restored for measured repros, but heavy-query execution time is not proved faster.
+- Older whole-document secret scanning and extreme Unicode normalization can still
+  cause sub-second request-path stalls.
+- UUID ties can change ordering across separately ingested stores; same-store
+  retrieval remains deterministic for retained inputs.
+- Large documents take minutes to become fully ready: a 5 MB document
+  (4,999,000 bytes) took about 7 minutes (7m00.6s) to become ready with models off
+  and graph on.
+- Non-object document requests keep an INFO-level 422 detail wording difference;
+  HTTP status, error code, location, and OpenAPI schema remain unchanged.
 
 ## [0.9.1] - 2026-11-01
 
@@ -645,7 +673,9 @@ for missing run output or a signed-tag verification.
 - **Test Suite**: 159+ tests covering unit, integration, P0 hotfixes, and performance benchmarks.
 - **CI Pipeline**: GitHub Actions workflow with Black, Ruff, mypy, pytest + coverage, and Codecov upload.
 
-[Unreleased]: https://github.com/Yasou13/MESA/compare/v0.9.0...HEAD
+[Unreleased]: https://github.com/Yasou13/MESA/compare/v0.9.2...HEAD
+[0.9.2]: https://github.com/Yasou13/MESA/compare/v0.9.1...v0.9.2
+[0.9.1]: https://github.com/Yasou13/MESA/compare/v0.9.0...v0.9.1
 [0.9.0]: https://github.com/Yasou13/MESA/compare/v0.8.1...v0.9.0
 [0.8.1]: https://github.com/Yasou13/MESA/compare/v0.8.0...v0.8.1
 [0.8.0]: https://github.com/Yasou13/MESA/compare/v0.7.1...v0.8.0

README.md

@@ -7,7 +7,7 @@
 
 MESA is an LLM-independent, source-linked memory engine for AI applications. It stores documents with exact provenance (document, revision, passage) and returns verifiable, scoped evidence over HTTP, Python SDK, and MCP. MESA returns evidence; it never writes the final answer or hallucinates sources.
 
-Package version: **0.9.1**. V4 is the current API. Production qualification remains at the release candidate stage: **NO-GO** until the [release gates](https://github.com/Yasou13/MESA/blob/main/docs/release.md) pass.
+Package version: **0.9.2**. V4 is the current API. Production qualification remains at the release candidate stage: **NO-GO** until the [release gates](https://github.com/Yasou13/MESA/blob/main/docs/release.md) pass.
 
 ## Installation
 

docker-compose.v4.yml

@@ -5,7 +5,7 @@ x-mesa-v4-runtime: &mesa-v4-runtime
     context: .
     dockerfile: Dockerfile
     target: runtime
-  image: ${MESA_V4_IMAGE:-mesa-memory:0.9.1}
+  image: ${MESA_V4_IMAGE:-mesa-memory:0.9.2}
   restart: "on-failure:5"
   read_only: true
   tmpfs:

docs/README.md

@@ -2,7 +2,7 @@
 
 Bu klasörde 12 kullanım/geliştirme rehberi ve 8 mimari karar belgesi vardır.
 Hepsini sırayla okumak gerekmez. Amacınıza göre aşağıdaki giriş noktasını seçin.
-Paket sürümü 0.9.1; V4 güncel API'dir.
+Paket sürümü 0.9.2; V4 güncel API'dir.
 
 ## Nereden başlamalıyım?
 

docs/RUNBOOK.md

@@ -245,7 +245,7 @@ audit must remain `NOT EXECUTED`, never converted to a simulated success.
 
 ## Upgrading to v0.8.0
 
-Current package metadata is 0.9.1. The v0.8.1 publication was owner-approved with the limitations in
+Current package metadata is 0.9.2. The v0.8.1 publication was owner-approved with the limitations in
 the [release notes](../CHANGELOG.md#v081-publication); it does not certify production
 readiness. Confirm actual tags in GitHub Releases. This guide describes changes
 since v0.7.1, not a completed production migration.

docs/api-reference.md

@@ -1,6 +1,6 @@
 # API reference
 
-MESA v4 tenant/dataset yetkili versioned API sunar. Paket 0.9.1; yayın sahibinin adımıdır ve production sertifikasyonu içermez.
+MESA v4 tenant/dataset yetkili versioned API sunar. Paket 0.9.2; yayın sahibinin adımıdır ve production sertifikasyonu içermez.
 
 Versioned memory, health ve metrics çağrıları `X-API-Key` ister. `/health/init`, OpenAPI, Swagger ve Redoc public yüzeylerdir; ağ/reverse-proxy politikası gerekir. V4 anahtarı
 `mesa-v4-admin issue-key` ile üretilir; yalnız saltlı scrypt özeti saklanır.

docs/architecture-v4.md

@@ -1,6 +1,6 @@
 # MESA Core current V4 architecture
 
-> **Package version:** 0.9.1, without production certification.
+> **Package version:** 0.9.2, without production certification.
 
 This document owns the current V4 design; production qualification remains
 release candidate / `NO-GO` under the [release policy](release.md).

docs/concepts-and-ingestion.md

@@ -98,3 +98,9 @@ Canonical V4 insertion requires the combined processing consumer. A combined
 runtime without models stores documents and finds them with BM25; vector search
 starts when embedding is enabled (`degraded_sources` shows `vector` until then);
 503 `canonical_processing_unavailable` happens only on a non-combined profile.
+
+Document ingestion briefly waits for queue capacity for the current chunk,
+keeping its chunk plan within the request. Persistent pressure still returns
+`503 queue_over_capacity`; retry the exact payload to resume stored ordinals.
+SDK `add_document(deadline=600)` uses the remaining document budget for this
+HTTP request, allowing a large upload to wait for capacity.

docs/configuration.md

@@ -45,7 +45,12 @@ chunking mode, ordinary article chunks inherit the document's law number from
 `metadata.law_number` or a `Kanun Numarası`, `Kanunun Numarası` or `Kanun No.`
 header in the first 3,000 characters. Continuation pieces keep that article's
 identity; preambles, GEÇİCİ/EK articles and plain plans receive none. Setting the
-switch to false disables automatic identity assignment for new chunks. Stored
+switch to false disables automatic identity assignment for new chunks. It also
+disables legal processing during search, including citation resolution, the
+exact-match slot, decision künye promotion, legal ranking factors and graph
+citation parsing. Ordinary lexical, vector and graph retrieval still run.
+Search ignores stored legal identities and item snapshots while the switch is
+off and omits their legal result annotations; it does not erase them. Stored
 identities are immutable on resume. Any 3–5 digit law number is accepted by the
 typed identity API; known numbers retain their existing statute code (6098 →
 TBK). Unknown law names and inconsistent number/code pairs are rejected.
@@ -60,8 +65,9 @@ amount, duration and outcome items. Search returns them in
 offsets refer to the original chunk, including when a result quotes a shorter
 extracted fact. Items remain separate from SPO facts and do not enter fusion
 scores. Decision references can use the bounded exact-match slot below.
-Disabling the switch prevents new item snapshots; existing
-snapshots remain. Purging a document erases these source quotes too.
+Disabling the switch prevents new item snapshots and hides existing snapshots
+from search results. Existing snapshots remain stored. Purging a document
+erases these source quotes too.
 
 An exact law-and-article query puts at most three matching chunks first, in
 document order. Other results retain their fused order (or reranked order when

docs/reference/openapi.json

@@ -2,7 +2,7 @@
   "openapi": "3.1.0",
   "info": {
     "title": "MESA API",
-    "version": "0.9.1"
+    "version": "0.9.2"
   },
   "paths": {
     "/v4/catalog/workspaces": {

docs/release.md

@@ -1,6 +1,6 @@
 # Release procedure
 
-V4 paket sürümü `0.9.1`; yayın aşağıdaki “Yayınlama”
+V4 paket sürümü `0.9.2`; yayın aşağıdaki “Yayınlama”
 adımlarıyla depo sahibinindir. Aşağıda kayıtlı yayın istisnasını depo sahibi
 v0.8.1 source release için açıkça onayladı. Production qualification halen
 release candidate olarak değerlendirilir. Production kararı

mesa_api/v4_router.py

@@ -1435,24 +1435,37 @@ def matches(plan: list[str]) -> bool:
                 if ordinal in admitted:
                     continue
                 chunk_id = f"{revision_id}-c{ordinal:05d}"
-                response = await insert_memory(
-                    request,
-                    V4MemoryInsertRequest(
-                        **payload.model_dump(exclude={"content"}),
-                        content=chunk,
-                        legal_identity=identities[ordinal],
-                        revision_id=revision_id,
-                        chunk_id=chunk_id,
-                        revision_number=revision["revision_number"],
-                        chunk_ordinal=ordinal,
-                        supersedes_revision_id=revision["supersedes_revision_id"],
-                        finalize_revision=ordinal == len(chunks) - 1,
-                        idempotency_key=chunk_id,
-                    ),
-                    dao,
-                    access_control,
-                    document_chunk=True,
+                chunk_payload = V4MemoryInsertRequest(
+                    **payload.model_dump(exclude={"content"}),
+                    content=chunk,
+                    legal_identity=identities[ordinal],
+                    revision_id=revision_id,
+                    chunk_id=chunk_id,
+                    revision_number=revision["revision_number"],
+                    chunk_ordinal=ordinal,
+                    supersedes_revision_id=revision["supersedes_revision_id"],
+                    finalize_revision=ordinal == len(chunks) - 1,
+                    idempotency_key=chunk_id,
                 )
+                capacity_deadline = asyncio.get_running_loop().time() + 5.0
+                while True:
+                    try:
+                        response = await insert_memory(
+                            request, chunk_payload, dao, access_control,
+                            document_chunk=True,
+                        )
+                        break
+                    except HTTPException as exc:
+                        if (
+                            exc.status_code != 503
+                            or exc.detail != "queue_over_capacity"
+                            or asyncio.get_running_loop().time() >= capacity_deadline
+                            or await request.is_disconnected()
+                        ):
+                            raise
+                        # Retain the plan and current ordinal while the durable
+                        # consumer makes room. Other errors keep their contract.
+                        await asyncio.sleep(0.05)
                 admitted[ordinal] = response["mutation_id"]
             return {
                 "document_id": payload.document_id,

mesa_client/client.py

@@ -1,5 +1,5 @@
 """
-MESA v0.9.1 Client SDK
+MESA v0.9.2 Client SDK
 
 A robust, type-safe Python client for the MESA memory system.
 Provides both synchronous and asynchronous implementations utilizing httpx.
@@ -262,7 +262,7 @@ def __init__(
     ):
         self.base_url = base_url.rstrip("/")
         self.max_retries = max_retries
-        headers = {"User-Agent": "mesa-client/v0.9.1"}
+        headers = {"User-Agent": "mesa-client/v0.9.2"}
         if api_key:
             headers["X-API-Key"] = api_key
 
@@ -327,7 +327,7 @@ def __init__(
     ):
         self.base_url = base_url.rstrip("/")
         self.max_retries = max_retries
-        headers = {"User-Agent": "mesa-client/v0.9.1"}
+        headers = {"User-Agent": "mesa-client/v0.9.2"}
         if api_key:
             headers["X-API-Key"] = api_key
 
@@ -779,7 +779,10 @@ def add_document(
         backoff = 0.5
         while True:
             try:
-                return self._request("POST", "/v4/documents", json=payload)
+                return self._request(
+                    "POST", "/v4/documents", json=payload,
+                    timeout=max(0.001, total_deadline - (time.monotonic() - start)),
+                )
             except MesaAPIError as exc:
                 if (
                     exc.status_code == 503
@@ -1230,7 +1233,10 @@ async def add_document(
         backoff = 0.5
         while True:
             try:
-                return await self._request("POST", "/v4/documents", json=payload)
+                return await self._request(
+                    "POST", "/v4/documents", json=payload,
+                    timeout=max(0.001, total_deadline - (time.monotonic() - start)),
+                )
             except MesaAPIError as exc:
                 if (
                     exc.status_code == 503

mesa_mcp/__init__.py

@@ -2,4 +2,4 @@
 
 __all__ = ["__version__"]
 
-__version__ = "0.9.1"
+__version__ = "0.9.2"

mesa_memory/api/server.py

@@ -911,7 +911,7 @@ def _canonical_readiness_failures(health: dict[str, object]) -> list[str]:
 
 @app.get("/health", dependencies=[Depends(get_api_key)])
 async def health():  # type: ignore[no-untyped-def]
-    health = await state.dao.health_check()
+    health = await state.dao.health_check(detailed=False)
     rebuild = health.get("v4_rebuild") or {}
     backend_healthy = all(
         health.get(component, {}).get("status") in {"healthy", "not_initialized"}

mesa_memory/document_chunking.py

@@ -31,12 +31,12 @@
 })
 _TRAILING_WORD = re.compile(r"([^\W_]+)$")
 _ARTICLE_HEADING = re.compile(
-    r"^[ \t]*(?:(?:GEÇİCİ|EK)[ \t]+)?MADDE[ \t]+\d+"
-    r"(?:/[A-ZÇĞİÖŞÜ]+)?(?:[ \t]+ila[ \t]+\d+)?[ \t]*[-–—−:]",
+    r"^[ \t]*+(?:(?:GEÇİCİ|EK)[ \t]++)?MADDE[ \t]++\d++"
+    r"(?:/[A-ZÇĞİÖŞÜ]++)?(?:[ \t]++ila[ \t]++\d++)?[ \t]*+[-–—−:]",
     re.MULTILINE | re.IGNORECASE,
 )
 _CASE_LAW_PREAMBLE = re.compile(
-    r'^[ \t]*["“]?İçtihat Metni["”]?[ \t]*\r?$', re.MULTILINE | re.IGNORECASE
+    r'^[ \t]*+["“]?İçtihat Metni["”]?[ \t]*+\r?$', re.MULTILINE | re.IGNORECASE
 )