Define and enforce data boundaries for production use

[JoshuaChi]

Context

d-engine currently has no documented or enforced limits on key size, value size,
or total dataset size. Users have no way to know what d-engine can safely hold,
and d-engine has no way to fail fast when those limits are exceeded.

We also have preliminary resource footprint estimates (binary size, memory, disk)
that need to be verified with real measurements before publishing.

---

Goals

1. Measure: value size vs performance
   Benchmark write latency and throughput across value sizes:
   1KB / 10KB / 100KB / 1MB
   Record where latency degrades and throughput drops.
   This drives the max value size decision.

2. Measure: resource footprint (verify estimates)
   The following are estimates from internal analysis — must be confirmed
   with real runs on Linux x86_64 release build before publishing:

   Metric	Estimate	Verified
   Binary size (stripped)	~8–10 MB	☐
   Memory idle (3-node)	70–120 MB/node	☐
   Memory under load ~10K ops/s	150–250 MB/node	☐
   Disk fresh start	~5 MB	☐
   Disk after benchmark load	~62–67 MB	☐
   Min viable: 0.5 core / 128MB RAM / 500MB disk	unverified	☐

   Test environment to document: instance type, OS, d-engine version,
   RocksDB config, snapshot enabled/disabled.

3. Enforce: input guards at API entry
   Add size validation at the put() entry point:

   • max key size (proposed: 512 bytes)
   • max value size (proposed: 1MB, confirm after benchmark in Goal 1)
   • return explicit error when exceeded

4. Document: publish verified numbers
   Once measured, add to README and docs:

   • Recommended operating ranges
   • Resource footprint table (verified, not estimated)
   • Explicit warning: snapshot disabled by default → unbounded log growth
     in production. Must be called out clearly.

---

Proposed limits (confirm after Goal 1)

Dimension	Proposed	Notes
Max key size	512 bytes	Coordination keys are short
Max value size	1 MB	Confirm with perf curve
Recommended dataset	≤ 8 GB	Coordination metadata, not DB

---

Out of scope

• Changing storage backend behavior
• Per-namespace or per-key-prefix limits
• Dynamic limit configuration