Compose Texo as a BatPak scale-out system - #11
Conversation
Add identity-preserving exact forks, resumable imported read models, and an atomic replica ledger compatibility boundary for batpak#227. Open replica hosts through BatPak ReadOnly so reads never advance the physical frontier.
|
Important Review skippedToo many files! This PR contains 55 files, which is 5 over the limit of 50. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (4)
📒 Files selected for processing (56)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b06eeafdc9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let socket = | ||
| crate::compat::netbat::private_socket_addr(&addr).map_err(|error| TexoError::Config { | ||
| detail: format!("replica listener {addr}: {error}"), | ||
| source: None, | ||
| })?; |
There was a problem hiding this comment.
When texo serve runs under the provided systemd deployment, TEXO_AGENT_ADDR defaults to 0.0.0.0:8787 in deploy/env.example, but this path validates the normal HTTP listener with private_socket_addr, which rejects wildcard/public endpoints as if they were plaintext replica clients. That makes the deployed service fail during startup before binding the listener; the private-address restriction should be applied only to replica netbat endpoints, not the public HTTP server bind address.
Useful? React with 👍 / 👎.
| Capability::Network { | ||
| policy: NetPolicy::DenyAll, | ||
| }, |
There was a problem hiding this comment.
When bootstrap sees TEXO_LLM_API_KEY, it writes texo extract as the configured extractor, and the deploy env configures hosted DashScope models; that extractor needs outbound network on cache misses. Running it under this helper with NetPolicy::DenyAll makes normal session/ingest extraction fail for uncached documents even though the model key is configured, so the confinement policy needs a path for the intended hosted extractor or bootstrap should not select it.
Useful? React with 👍 / 👎.
| if value.is_empty() | ||
| || value.len() > 128 | ||
| || !value | ||
| .bytes() | ||
| .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.')) |
There was a problem hiding this comment.
A topology can currently declare a journal id of . or .. because dots are allowed and there is no special-case rejection. Replica cursor/evidence paths later join the journal id as a directory component, so texo replica follow .. can write outside the intended .texo/replication/<workspace>/<journal>/ subtree. The validator should reject ./.. (and probably dot-only path components) as the doc comment promises for path-like ids.
Useful? React with 👍 / 👎.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| @@ -25,7 +29,7 @@ use crate::config::WorkspaceConfig; | |||
| use crate::error::TexoError; | |||
There was a problem hiding this comment.
Changing the only accepted manifest schema to texo.backup.v2 leaves no reader or migration path for backups created as texo.backup.v1. After upgrading, verification rejects those existing backups before restore, so an operator cannot recover data from a previously valid Texo backup.
Context Used: AGENTS.md (source)
Artifacts
- Contains supporting evidence from the run (text/x-shellscript; charset=utf-8).
Repro: runtime output showing v1 verification and restore rejection
- Keeps the command output available without making the summary code-heavy.
Repro: focused backup suite output confirming the current v2 path succeeds
- Keeps the command output available without making the summary code-heavy.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/backup.rs
Line: 29
Comment:
**Existing Backups Become Unrestorable**
Changing the only accepted manifest schema to `texo.backup.v2` leaves no reader or migration path for backups created as `texo.backup.v1`. After upgrading, verification rejects those existing backups before restore, so an operator cannot recover data from a previously valid Texo backup.
**Context Used:** AGENTS.md ([source](https://app.greptile.com/heyoub/github/freebatteryfactory/texo/-/custom-context?memory=7a12442e-73af-4ed1-b813-3fb795a7bbff))
How can I resolve this? If you propose a fix, please make it concise.| .get(&manifest.workspace_id) | ||
| .cloned() | ||
| .ok_or_else(|| backup_error("backup config does not contain its workspace"))?; | ||
| workspace.store_path = crate::config::WorkspaceEntry::for_id(&manifest.workspace_id).store_path; | ||
| let restore_store_path = crate::config::WorkspaceEntry::for_id(&manifest.workspace_id) | ||
| .primary() | ||
| .map_err(|error| backup_error(error.to_string()))? | ||
| .store_path; | ||
| workspace |
There was a problem hiding this comment.
This reconstructs the workspace with WorkspaceEntry::for_id instead of restoring the backed-up entry. A workspace with custom extraction, semantic settings, document globs, or named replica journals is restored with defaults, so later ingestion can produce different results and clients pinned to those journals fail to start.
Context Used: AGENTS.md (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/backup.rs
Line: 374-381
Comment:
**Restore Replaces Workspace Configuration**
This reconstructs the workspace with `WorkspaceEntry::for_id` instead of restoring the backed-up entry. A workspace with custom extraction, semantic settings, document globs, or named replica journals is restored with defaults, so later ingestion can produce different results and clients pinned to those journals fail to start.
**Context Used:** AGENTS.md ([source](https://app.greptile.com/heyoub/github/freebatteryfactory/texo/-/custom-context?memory=7a12442e-73af-4ed1-b813-3fb795a7bbff))
How can I resolve this? If you propose a fix, please make it concise.| if let Some(cursor) = unchanged_local_cursor(previous, source_ceiling) { | ||
| return Ok(unchanged_report(root, circuit, cursor)); |
There was a problem hiding this comment.
When a source store is replaced by different history with the same final sequence, this shortcut returns the old cursor before comparing its recorded event identity with the current source. The replica is then reported as refreshed and can serve stale history until the replacement source advances.
Context Used: AGENTS.md (source)
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/replication.rs
Line: 313-314
Comment:
**Unchanged Frontier Skips Source Validation**
When a source store is replaced by different history with the same final sequence, this shortcut returns the old cursor before comparing its recorded event identity with the current source. The replica is then reported as refreshed and can serve stale history until the replacement source advances.
**Context Used:** AGENTS.md ([source](https://app.greptile.com/heyoub/github/freebatteryfactory/texo/-/custom-context?memory=7a12442e-73af-4ed1-b813-3fb795a7bbff))
How can I resolve this? If you propose a fix, please make it concise.1 participant
What changed
This composes Texo through the BatPak 0.10 family as a scale-out system instead of treating one physical store as the product boundary.
texo install --client allprovision independent Codex, Claude, and Cursor read replicas and refresh them before MCP servesdocs/BATPAK-SCALEOUT-COMPOSITION.mdWhy
BatPak's single-writer contract is per physical materialization, not a global scaling ceiling. Texo now uses the intended coordinate/CQRS model: canonical logs own ordered authority-bearing writes; independent stores materialize read models and serve concurrently. Replica lag, identity, anchors, and authority are explicit rather than inferred.
Neutral BatPak 0.10 gaps remain behind deletable typed shims. Upstream replacements are tracked in freebatteryfactory/batpak_DEPRECATED#227 through #233; no gap waters down the product path.
Security and failure behavior
Validation
just verify: fmt, clippy-D warnings, cargo-deny, typos, hygiene, and full suite greenjust test-invariants: projection laws 10/10, compile-fail 1/1, BatPak family spikes 7/7/tmpescape attempt creates no file and no completion markerNo paid model calls were made by the new tests. The immutable hackathon tag and
mainwere not modified.Greptile Summary
This PR turns Texo into a journal-aware BatPak scale-out system. The main changes are:
Confidence Score: 4/5
Backup recovery and unchanged replica refresh need fixes before merging.
Existing backups can be rejected after upgrading. Restore can silently discard saved workspace behavior and journal routes. A same-length source replacement can be reported as refreshed while stale data remains.
src/backup.rs and src/replication.rs
What T-Rex did
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR C[Canonical journal] -->|paged import| R1[Codex replica] C -->|paged import| R2[Claude replica] C -->|paged import| R3[Cursor replica] C -->|exact fork| F[Identity-preserving fork] R1 --> M1[MCP] R2 --> M2[MCP] R3 --> M3[MCP] C --> B[Backup envelope] B --> V[Offline verification] V --> W[Restored workspace]%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% flowchart LR C[Canonical journal] -->|paged import| R1[Codex replica] C -->|paged import| R2[Claude replica] C -->|paged import| R3[Cursor replica] C -->|exact fork| F[Identity-preserving fork] R1 --> M1[MCP] R2 --> M2[MCP] R3 --> M3[MCP] C --> B[Backup envelope] B --> V[Offline verification] V --> W[Restored workspace]Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "fix(confinement): make runtime roots rea..." | Re-trigger Greptile
Context used: