perf(storage): add bulk plugin-storage upserts

[aryasaatvik]

Summary

Adds a reusable bulk upsert path for FumaDB and routes plugin-storage bulk writes through adapter-level conflict upserts instead of delete-then-create batches.

This keeps the public plugin-storage API small while giving storage-heavy plugins a faster and safer write path for repeated key updates.

Implementation Shape

FumaDB Query Surface

packages/core/fumadb/src/query/index.ts

AbstractQuery
  -> upsert(table, options)
  -> upsertMany(table, options)

upsertMany mirrors the existing single-row upsert shape, but accepts many values and forwards through the same ORM policy boundary as other write operations.

ORM Adapter Contract

packages/core/fumadb/src/query/orm/index.ts

ORMAdapter
  -> upsert?(table, options)
  -> upsertMany?(table, options)

toORM(adapter)
  -> apply write policy constraints
  -> reject unsupported adapter capabilities loudly
  -> forward bulk upsert to adapter

The ORM wrapper remains the policy boundary. Callers do not bypass table policy checks by calling adapter methods directly.

Adapter Implementations

memoryAdapter
  -> upsertMany(table, { target, update, values })
  -> find matching row by conflict target
  -> update configured columns or insert new row

drizzleAdapter
  -> upsertMany(table, { target, update, values })
  -> insert(values)
  -> onConflictDoUpdate({ target, set })

The memory adapter is the reference implementation for semantic behavior. The Drizzle adapter maps the same contract to SQL conflict upserts.

Plugin Storage Facade

packages/core/sdk/src/plugin-storage.ts

PluginStorageFacade
  -> putMany({ owner, entries })
  -> removeMany({ owner, entries })

PluginStorageCollectionFacade
  -> putMany({ owner, entries })
  -> removeMany({ owner, keys })

Collection helpers keep plugin authors on collection-local keys while the lower-level facade can still batch entries across collections.

Bulk Write Flow

pluginStorage.collection(def).putMany({ owner, entries })
  -> attach collection name to every entry
  -> dedupe entries by plugin storage id
  -> compute tenant, owner, and subject partition
  -> core.upsertMany("plugin_storage", { target, update, values })
  -> FumaDB AbstractQuery.upsertMany
  -> ORM wrapper applies write policy constraints
  -> adapter performs conflict upsert

Policy And Ownership Flow

putMany(owner: "org")
  -> subject = __org__
  -> rows written to org partition

putMany(owner: "user")
  -> requires executor subject
  -> rows written to user partition

read path
  -> existing owner precedence stays unchanged
  -> user rows shadow org rows for visible reads

Scope Notes

• This PR is reusable storage infrastructure.
• It intentionally does not add JSON aggregation or keyset pagination.
• It intentionally does not rewrite semantic-search consumers.
• Unsupported adapters fail loudly instead of silently falling back to slower behavior.

Validation

• bun run --cwd packages/core/fumadb typecheck
• bun run --cwd packages/core/sdk typecheck
• bun run --cwd packages/core/fumadb test src/query/table-policy.test.ts
• bun run --cwd packages/core/sdk test src/plugin-storage.test.ts
• git diff --check upstream/main...HEAD
• touched-file oxfmt --check
• touched-file oxlint -c .oxlintrc.jsonc --deny-warnings

Follow-up Scope

JSON-document aggregation and keyset pagination are separate storage/query work and intentionally remain out of this PR.

[greptile-apps]

Greptile Summary

This PR adds a upsertMany primitive to FumaDB's query/adapter stack and routes plugin-storage bulk writes through conflict-aware INSERT … ON CONFLICT DO UPDATE statements, replacing the previous delete-then-insert pattern. Policy enforcement (create + update) is applied per-row at the ORM layer before any adapter call, and permitted rows are grouped by their policy condition key for efficient native batching.

• FumaDB layer: AbstractQuery.upsertMany is fully wired through the ORM (with policy grouping), drizzle adapter (native PG/SQLite with parameter-aware batching; per-row fallback for other providers), and memory adapter.
• SDK layer: putManyImpl in executor.ts drops the old delete+createMany loop in favour of a single upsertMany call; created_at is included in values but excluded from the update list, correctly preserving original timestamps on re-writes.
• Ergonomics: Collection-scoped putMany/removeMany helpers are added to PluginStorageCollectionFacade, removing the need for callers to manually inject the collection name.

Confidence Score: 5/5

Safe to merge; the change is well-scoped and all policy-enforcement paths are covered by the new tests.

Policy validation (both create and update) is enforced at the ORM layer before any adapter call. Empty-target and empty-update guards exist at both ORM and adapter levels. The native PG/SQLite path uses correct excluded.column references and parameter-aware batching. The ORM fallback for adapters without native upsertMany is logically sound. Cross-tenant write rejection and invalid-shape rejection are both tested. No correctness gaps were found in the changed paths.

No files require special attention. test-config.ts bypasses the ORM fallback when calling upsertMany on the lazy test DB, but this is benign since the memory adapter always provides the method.

Important Files Changed

Filename	Overview
packages/core/fumadb/src/query/orm/index.ts	Core ORM upsertMany implementation: applies per-row create/update policies, groups permitted rows by policy condition key, then dispatches to native adapter batch path or per-row upsert fallback. Logic is correct; conditionKey serialization is stable for deterministic policy output.
packages/core/fumadb/src/adapters/drizzle/query.ts	Adds native INSERT … ON CONFLICT DO UPDATE for PostgreSQL/SQLite with parameter-aware batch sizing, plus a per-row upsert fallback for other providers. Helper countConditionParameters correctly handles all Condition variants for batch-size arithmetic.
packages/core/fumadb/src/adapters/memory/index.ts	Memory adapter upsertMany: validates target/update are non-empty, iterates values, matches existing row by both v.where and target column equality, patches or creates. Correctly handles previous vacuous-truth concern by guarding target.length.
packages/core/sdk/src/executor.ts	Routes plugin-storage putMany through upsertMany instead of delete+createMany. created_at is correctly included in values (for new rows) but excluded from the update list, preserving original timestamps on re-writes. Adds collection-scoped putMany/removeMany helpers.
packages/core/fumadb/src/query/index.ts	Adds upsertMany to the AbstractQuery interface with correct generic typing over table/column names. No where clause exposed publicly — policy constraints are handled internally by the ORM layer.
packages/core/fumadb/src/query/table-policy.test.ts	Adds focused policy tests: bulk upsert that spans tenants fails fast; invalid target/update shapes throw; intra-tenant bulk upsert correctly updates existing and inserts new rows.
packages/core/sdk/src/plugin-storage.ts	Adds PluginStorageCollectionPutManyInput, PluginStorageCollectionRemoveManyInput interfaces and extends PluginStorageCollectionFacade with typed putMany/removeMany. Types are consistent with the executor implementation.
packages/core/sdk/src/plugin-storage.test.ts	Updates test plugin to use the new collection-scoped ctx.storage.toolCalls.putMany/removeMany API, removing the manual collection-name mapping. Cleaner ergonomics with no logic change.
packages/core/sdk/src/fuma-runtime.ts	Threads upsertMany through makeSafeFumaQuery by delegating to db.upsertMany — consistent with how other operations (upsert, updateMany) are wired.
packages/core/sdk/src/test-config.ts	Adds upsertMany to the lazy test DB shim with a guard against adapters that don't implement it at the adapter level. The guard bypasses the ORM fallback path, but is benign in practice since the memory adapter always provides the method.
packages/plugins/openapi/src/sdk/store.test.ts	Adds stub putMany/removeMany implementations (Effect.void) to the OpenAPI store mock to satisfy the updated PluginStorageCollectionFacade interface.
.changeset/plugin-storage-bulk-upserts.md	Patch changeset for both fumadb and sdk packages; description accurately summarises the change.

_{Reviews (2): Last reviewed commit: "fix(fumadb): validate bulk upsert confli..." | Re-trigger Greptile}

[aryasaatvik]

Additional downstream context from my fork: this bulk plugin-storage primitive is what makes storage-heavy plugins practical without hand-rolled adapter paths.

High-level shape:

plugin collection putMany/removeMany
  -> plugin-storage facade
  -> FumaDB upsertMany/deleteMany
  -> adapter conflict upsert
  -> storage-heavy plugin state update

AST outline of downstream consumers:

semanticSearchIndex
  -> jobs.putMany(...)
  -> chunks.removeMany(...)
  -> chunks.putMany(...)
  -> fingerprints.putMany(...)

openapiStore
  -> removeOperations(...)
  -> appendOperations(...)

Fork permalinks:

• Semantic-search job batch writes: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/plugins/semantic-search/src/sdk/tool-search-index.ts#L531-L548
• Semantic-search chunk replacement: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/plugins/semantic-search/src/sdk/tool-search-index.ts#L1240-L1267
• Marking chunks indexed in bulk: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/plugins/semantic-search/src/sdk/tool-search-index.ts#L1412-L1429
• Fingerprint upserts: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/plugins/semantic-search/src/sdk/tool-search-index.ts#L1542-L1551
• OpenAPI operation store bulk replace: https://github.com/aryasaatvik/executor/blob/7784c0f8512564997a55d3ab46a1d84303f6d4ef/packages/plugins/openapi/src/sdk/store.ts#L159-L175

Call stack:

index scan
  -> compute changed jobs/chunks/fingerprints
  -> collection.putMany(...)
  -> pluginStorage.putMany(...)
  -> core.upsertMany("plugin_storage", ...)
  -> FumaDB adapter upsertMany

This PR intentionally does not include semantic search. It adds the storage primitive that lets semantic search, OpenAPI operation refresh, and other plugin-maintained indexes use one safe SDK path.