Repository navigation
ci: run Go test packages serially (fix Windows throughput failure) - #5
Closed
ChromaBeast wants to merge 21 commits into
Closed
ChromaBeast wants to merge 21 commits into
ChromaBeast wants to merge 21 commits into
Conversation
Values up to 2 MiB, page-slot reuse, and WAL frame cap
Hand-written text exposition (no new deps). Authenticated by admin session or bearer token (--api-token / BEASTDB_METRICS_TOKEN); no loopback bypass. Adds buffer-pool hit/miss counters and checkpoint timestamps, gRPC metrics interceptors, runbook docs with scrape config and alert rules.
WAL gains Append (assign LSN, no fsync), WaitDurable/Sync with a shared syncedLSN so one fsync covers all appended records, and a sticky fsync error. Write keeps its signature and durability but now group-commits too. Engine writes (Put/PutIfAbsent/PutIf/Delete/DeleteIf/BatchWrite[If] and replicated apply) go through a commit queue: a committer goroutine runs a group of writes under the engine lock (append + apply), issues one fsync, notifies the observer in LSN order, then releases the lock and acks writers. Reads and replication never see a write before it is durable. A failed fsync halts the engine and fails every write in the group. Concurrent Put benchmark: 7.82 ms/op -> 0.51 ms/op (~15x).
Group commit applies writes to pages before the group's fsync, so the buffer pool could flush a page ahead of its WAL records. Add a WAL barrier (WAL.EnsureDurable) called before eviction, FlushPage and FlushAll; a failing barrier blocks the page write. Adds regression tests and the upgrade spec.
Resolved conflicts in engine.go and buffer_pool.go by keeping both the group-commit/WAL-barrier fields and the metrics counters.
…ench and Studio design
… the 200-line limit No behaviour change: functions moved verbatim into wal_sync.go, wal_lifecycle.go, buffer_pool_wal.go, buffer_pool_stats.go, runner_load.go and runner_run.go.
…guideline About 400 lines is a prompt to ask whether a file does more than one thing, not a gate. Generated code, tests and data are exempt. Updates GEMINI.md (Antigravity), CONTRIBUTING.md and demo.ps1, and adds AGENTS.md (Codex) and CLAUDE.md (imports AGENTS.md).
…erformance-3d83f3 Group commit, gRPC roles, /metrics, beastbench and security policy
…, and new sign-in - Adopt the Figma dark palette; replace the top header with a sidebar (Overview, Table Editor, Operations, API Keys & Roles) - Overview: buffer-pool occupancy map and WAL head with LSN flash - Table Editor: 64-bit key bit matrix per row; flatter cards and tables - Sign-in: brand-led layout with page-grid backdrop - Rebuild the embedded static bundle
…s it DiskFreeBytes is unix-only, so TestMetricsAuthAndFormat failed on Windows CI with a missing beastdb_disk_free_bytes line. Require the gauge only when DiskFreeBytes reports support. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Instead of waking a dedicated committer goroutine for every write, the writer that finds no commit in progress runs the queued group on its own goroutine; if more writers queued meanwhile, a drain goroutine serves them. Group commit semantics are unchanged (one fsync per group, lock held across it, observer in LSN order). Aims to remove a cross-goroutine wake-up per uncontended write, which can cost up to a timer tick on Windows. Not yet verified on Windows. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Fix Windows CI after #2: metrics test, single-writer commit path
… quiet disk On windows-latest the full-suite run measured TestEngineSystemWorkloadBenchmark at ~315 ops/s with default package parallelism, ~1650-1830 with -p 1, and the same test run alone is ~2500-3000 at every commit since before the upgrade work. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Owner
Author
|
Superseded by #6, which fixes the root cause (per-page fsyncs during eviction slowing parallel test packages) instead of serializing packages with -p 1. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
TestEngineSystemWorkloadBenchmark(fixed 1000 ops/s floor) fails onwindows-latestin the full suite. Diagnosed with temporary A/B runs on the same Windows runner:cmd/beastbench-p 1So the engine is not slower in isolation and
beastbenchis not the cause; the benchmark is fsync-bound and loses to other packages hitting the disk in parallel.Change
CI runs
go test -p 1;AGENTS.mdnote updated. The floor is unchanged.Not explained
Head degrades more under contention than the old base did (~320 vs ~900-1300). Isolation shows no regression, so this PR treats the symptom; the sensitivity itself is still worth a look.
🤖 Generated with Claude Code