Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .changeset/query-bundle-preload.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
"@contentrain/query": minor
---

feat(sdk): CDN bundle preload mode — collapse per-render fetch waterfalls into a single conditional GET

**@contentrain/query**: `createContentrain({ bundle: true })` makes `HttpTransport` fetch `_bundle/{locale}.json` (one artifact with every JSON model for the locale) and serve all collection/singleton/dictionary/document-index reads from memory. Default off — behavior without `bundle` is unchanged.

- **Opt-in `bundle` config.** `true` → `{ revalidateMs: 60_000 }`; the bundle is revalidated with a conditional request (unchanged → `304`) after `revalidateMs`.
- **Transparent fallback.** Bundle `404`/invalid/network error → per-path fetching (retried after `revalidateMs`), so the SDK can ship before the CDN publishes bundles.
- **Scoped coverage.** Only `content/`, `documents/`, `meta/` paths consult the bundle; document bodies, `withMeta()`, manifests, and `models/*` keep per-path fetch. Stale primed paths are evicted when a fresh bundle drops them. Non-i18n `content/{model}/data.json` entries resolve via `defaultLocale`.
- **`client.preload(locale?)`** for eager warmup (e.g. SSR boot); resolves `true` when a bundle was found. Concurrent first reads dedupe into a single bundle request.
30 changes: 29 additions & 1 deletion docs/packages/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -513,14 +513,42 @@ const models = await client.models() // models/_index.json
const model = await client.model('faq') // models/faq.json
```

### Bundle Preload

Opt-in mode that collapses per-render fetch waterfalls (N content requests + include resolution) into a single conditional GET. The transport fetches `_bundle/{locale}.json` — one artifact containing every JSON model for that locale — and serves all content, document-index, and dictionary reads from memory:

```ts
const client = createContentrain({
projectId: '350696e8-...',
apiKey: 'crn_live_xxx',
bundle: true, // or { revalidateMs: 30_000 }
})

// Optional eager warmup (e.g. SSR boot) — resolves true when a bundle was found
await client.preload('en')

// Zero network: served from the primed bundle
const posts = await client.collection('faq').locale('en').all()
const t = await client.dictionary('ui').locale('en').get()
```

- **Coverage** — all JSON models (collections, singletons, dictionaries, document indexes) come from the bundle. Document bodies (`bySlug()`), entry metadata (`withMeta()`), and media manifests keep using per-path fetch.
- **Revalidation** — the bundle is re-checked after `revalidateMs` (default `60_000`) with a conditional request; an unchanged bundle answers `304` at negligible cost.
- **Fallback** — if the bundle is missing (`404`), invalid, or the request fails, the transport transparently falls back to per-path fetching and retries after `revalidateMs`. With `bundle` unset, behavior is identical to previous versions.
- **Non-i18n paths** — `content/{model}/data.json` entries ship in every locale bundle and are resolved via `defaultLocale` (default `'en'`).

::: info Requires bundle publishing
The `_bundle/{locale}.json` artifact is produced by Studio CDN publishing. Until the project's first rebuild publishes a bundle, the SDK silently uses per-path fetching — enabling `bundle: true` early is safe.
:::

### CDN vs Local Comparison

| Aspect | Local (`#contentrain`) | CDN (`createContentrain()`) |
|--------|----------------------|---------------------------|
| Data source | Bundled `.mjs` files | HTTP fetch from Studio CDN |
| Return type | Sync (`T[]`) | Async (`Promise<T[]>`) |
| Auth | None | API key required |
| Caching | In-memory (embedded) | ETag-based HTTP cache |
| Caching | In-memory (embedded) | ETag-based HTTP cache + optional bundle preload |
| Use case | SSG, build-time | SSR, client-side, serverless, mobile |

### Error Handling
Expand Down
28 changes: 27 additions & 1 deletion packages/sdk/js/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -333,14 +333,40 @@ const models = await client.models()
const model = await client.model('faq')
```

### Bundle Preload

Opt-in mode that collapses per-render fetch waterfalls (N content requests + include resolution) into a single conditional GET. The transport fetches `_bundle/{locale}.json` — one artifact containing every JSON model for that locale — and serves all content/document-index/dictionary reads from memory:

```ts
const client = createContentrain({
projectId: '350696e8-...',
apiKey: 'crn_live_xxx',
bundle: true, // or { revalidateMs: 30_000 }
})

// Optional eager warmup (e.g. SSR boot) — resolves true when a bundle was found
await client.preload('en')

// Zero network: served from the primed bundle
const posts = await client.collection('faq').locale('en').all()
const t = await client.dictionary('ui').locale('en').get()
```

Behavior:

- **Coverage** — all JSON models (collections, singletons, dictionaries, document indexes) are served from the bundle. Document bodies (`bySlug()`), entry metadata (`withMeta()`), and media manifests keep using per-path fetch.
- **Revalidation** — the bundle is re-checked after `revalidateMs` (default `60_000`) with a conditional request; an unchanged bundle answers `304` at negligible cost.
- **Fallback** — if the bundle is missing (`404`), invalid, or the request fails, the transport transparently falls back to per-path fetching and retries the bundle after `revalidateMs`. With `bundle` unset, behavior is byte-for-byte identical to previous versions.
- **Non-i18n paths** — `content/{model}/data.json` entries ship in every locale bundle and are resolved via `defaultLocale` (default `'en'`).

### CDN vs Local

| Aspect | Local (`#contentrain`) | CDN (`createContentrain()`) |
|--------|----------------------|---------------------------|
| Data source | Bundled `.mjs` files | HTTP fetch from CDN |
| Return type | Sync (`T[]`) | Async (`Promise<T[]>`) |
| Auth | None | API key required |
| Caching | In-memory (embedded) | ETag-based HTTP cache |
| Caching | In-memory (embedded) | ETag-based HTTP cache + optional bundle preload |
| Use case | SSG, build-time | SSR, client-side, serverless |

## CommonJS Usage
Expand Down
94 changes: 93 additions & 1 deletion packages/sdk/js/src/cdn/http-transport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,115 @@ interface CacheEntry {
etag: string
}

export interface TransportConfig {
baseUrl: string
projectId: string
apiKey: string
/** Bundle preload mode. `true` → `{ revalidateMs: 60_000 }` */
bundle?: boolean | { revalidateMs?: number }
defaultLocale?: string
}

interface BundleState {
fetchedAt: number
paths: Set<string>
missing: boolean
inflight: Promise<void> | null
}

interface BundlePayload {
version?: string
paths?: Record<string, unknown>
}

/** Paths eligible for bundle serving — everything else (manifests, models/*) has no locale segment. */
const BUNDLE_SCOPE = /^(content|documents|meta)\//

export class HttpTransport {
private _baseUrl: string
private _projectId: string
private _apiKey: string
private _cache = new Map<string, CacheEntry>()
private _bundleMode: boolean
private _revalidateMs: number
private _defaultLocale: string
private _primed = new Map<string, unknown>()
private _bundles = new Map<string, BundleState>()

constructor(config: { baseUrl: string; projectId: string; apiKey: string }) {
constructor(config: TransportConfig) {
this._baseUrl = config.baseUrl.replace(/\/+$/, '')
this._projectId = config.projectId
this._apiKey = config.apiKey
const b = config.bundle
this._bundleMode = !!b
this._revalidateMs = typeof b === 'object' && typeof b.revalidateMs === 'number' ? b.revalidateMs : 60_000
this._defaultLocale = config.defaultLocale ?? 'en'
}

buildUrl(path: string): string {
return `${this._baseUrl}/${this._projectId}/${path}`
}

/** Eager warmup (e.g. SSR boot). Resolves `true` when a bundle was found and primed. */
async preload(locale?: string): Promise<boolean> {
if (!this._bundleMode) return false
const l = locale ?? this._defaultLocale
await this._ensureBundle(l)
return !this._bundles.get(l)?.missing
}

async fetch<T>(path: string): Promise<T> {
if (this._bundleMode && BUNDLE_SCOPE.test(path)) {
await this._ensureBundle(this._localeOf(path))
if (this._primed.has(path)) return this._primed.get(path) as T
// out-of-scope path (doc body, meta, ...) → regular per-path fetch
}
return this._networkFetch<T>(path)
}

/** Locale from a path's last segment (sans `.json`). `data` (non-i18n) → defaultLocale bundle. */
private _localeOf(path: string): string {
const last = path.slice(path.lastIndexOf('/') + 1).replace(/\.json$/, '')
return last === 'data' ? this._defaultLocale : last
}

private _ensureBundle(locale: string): Promise<void> | void {
const state = this._bundles.get(locale)
if (state?.inflight) return state.inflight
if (state && Date.now() - state.fetchedAt < this._revalidateMs) return

const next: BundleState = state ?? { fetchedAt: 0, paths: new Set(), missing: false, inflight: null }
this._bundles.set(locale, next)
next.inflight = (async () => {
try {
// _networkFetch is conditional: unchanged bundle → 304 → cached payload (cheap)
const bundle = await this._networkFetch<BundlePayload>(`_bundle/${locale}.json`)
if (bundle?.version !== '1' || !bundle.paths || typeof bundle.paths !== 'object') {
next.missing = true
return
}
// Evict primed paths absent from the fresh bundle (deleted models must not serve stale)
const fresh = new Set(Object.keys(bundle.paths))
for (const p of next.paths) {
if (!fresh.has(p)) this._primed.delete(p)
}
for (const [p, body] of Object.entries(bundle.paths)) this._primed.set(p, body)
next.paths = fresh
next.missing = false
}
catch {
// 404 (bundle not built yet) or network error → per-path fallback until next revalidate
next.missing = true
}
finally {
next.fetchedAt = Date.now()
next.inflight = null
}
})()
return next.inflight
}

private async _networkFetch<T>(path: string): Promise<T> {
const url = `${this._baseUrl}/${this._projectId}/${path}`
const cached = this._cache.get(path)

Expand Down
7 changes: 7 additions & 0 deletions packages/sdk/js/src/cdn/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ export interface ContentrainCDNConfig {
apiKey: string
baseUrl?: string
defaultLocale?: string
/** Bundle preload mode. `true` → `{ revalidateMs: 60_000 }` */
bundle?: boolean | { revalidateMs?: number }
}

export type ContentrainCDNClient = ReturnType<typeof createContentrain>
Expand All @@ -21,6 +23,8 @@ export function createContentrain(config: ContentrainCDNConfig) {
baseUrl: config.baseUrl ?? 'https://studio.contentrain.io/api/cdn/v1',
projectId: config.projectId,
apiKey: config.apiKey,
bundle: config.bundle,
defaultLocale: config.defaultLocale,
})
const defaultLocale = config.defaultLocale

Expand Down Expand Up @@ -54,13 +58,16 @@ export function createContentrain(config: ContentrainCDNConfig) {
manifest: () => transport.fetch<unknown>('_manifest.json'),
models: () => transport.fetch<unknown[]>('models/_index.json'),
model: (id: string) => transport.fetch<unknown>(`models/${id}.json`),

preload: (locale?: string) => transport.preload(locale),
}
}

// Re-exports
export { ContentrainError } from './errors.js'
export type { CollectionDataSource, SingletonDataSource, DictionaryDataSource, DocumentDataSource } from './data-source.js'
export { HttpTransport } from './http-transport.js'
export type { TransportConfig } from './http-transport.js'
export { CdnCollectionQuery } from './collection-query.js'
export { CdnSingletonAccessor } from './singleton-accessor.js'
export { CdnDictionaryAccessor } from './dictionary-accessor.js'
Expand Down
Loading
Loading