Skip to content

feat: Add the file-based override source - #2053

Open
kinyoklion wants to merge 9 commits into
rlamb/overrides-node-eventsfrom
rlamb/overrides-node-file-source
Open

kinyoklion wants to merge 9 commits into
rlamb/overrides-node-eventsfrom
rlamb/overrides-node-file-source

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Summary

This PR is based on the events branch because it completes the feature that the earlier branches build up, and it should merge after them.

This change adds the file-based override source described by the OVERRIDE specification to the Node.js server SDK. It is configured as { type: 'file', paths: [...] } in the overrides property of the data system options, alongside the source object and factory forms that the earlier change added.

The source reads one or more JSON or YAML files in the file data source document format, with optional flags, flagValues, and segments members, and combines their entries in the configured order. Duplicate keys across files fail the reload by default, or keep the first file's entry with the ignore handling. Change detection is one of two modes. Polling, the default, examines the files once per second, with a one second minimum, and works on every filesystem. Watching reacts to change notifications for the directories that contain the files. Change detection is in place before the initial load, so a change made between the two is not missed.

A configured file that does not exist contributes no overrides, so a file can be created later and deleting a file removes its overrides. A file that exists but cannot be read or parsed fails that whole reload: the last good overrides stay in effect, the failure is logged once, and the source retries after one second. Every applied change is logged at Info level with the overrides in effect and what each file supplied, and an unchanged reload logs nothing. The initial load is part of starting the client: the promise that waitForInitialization returns settles after it, and an evaluation made before that reads the layer as it is at that moment.

An invalid configuration fails client construction, the way an invalid file initializer does: no file paths, an unrecognized change detection mode, or a platform without filesystem support. An unrecognized duplicate keys handling is logged and replaced by its default. A polling interval below the minimum is logged and raised to one second, and one that is not a finite number or exceeds 2147483 seconds is logged and replaced by the default.

The specification requires YAML support without extra configuration. The Node.js SDK package gains a dependency on the yaml package (2.9.1, already present in the lockfile as a transitive dependency of other workspaces) and supplies its parser to the shared code as a platform default. A configured yamlParser takes precedence. The shared server package has no new dependency.

The existing file data sources keep their current behavior. The override source has its own document handling (format detection by file extension, validation of the document shape, entries keyed by their map key with a key field inside an entry set to that map key, so a definition pasted under a new key overrides that key, value expansion to a flag that is on and serves the value by fallthrough, missing-file handling, retry) and none of it applies to the FDv1 FileDataSource or the FDv2 file data initializer.

The tests mirror the Go file source tests over a mock filesystem: initial load, YAML, ordered merge, duplicate handling, missing files that appear and disappear, the Info log on each change, quiet watching of an absent file, reloads in both modes, retention across a malformed edit with recovery through the retry, and close. A server-node test drives the source through the client on real files, including the built-in YAML parser.

SDK-3247


Note

Overview
Adds an experimental file-based flag override source configurable via dataSystem.overrides: { type: 'file', paths: [...] }. It loads JSON/YAML documents (same shape as file data sources: flags, flagValues, segments), merges multiple files in order, and hot-reloads via polling (default, 1s) or directory watching. Missing files contribute nothing; parse/duplicate-key failures keep the last good snapshot, log errors, and retry.

Shared server code introduces FileOverrideSource, override-specific document validation (parseOverrideDocument / fileOverrideSourcePolicy), and expanded createOverrideSource validation (paths required, change-detection modes, poll interval bounds). Node server SDK adds the yaml package and registers a default yamlParser on LDClientNode; apps can override with yamlParser in override options.

Evaluation behavior is unchanged except overrides sit on the read path: overrideAffected reasons, waitForInitialization waits for the initial override load, and evaluations can use overrides before LD init when present in the layer.

Reviewed by Cursor Bugbot for commit 0329a1f. Bugbot is set up for automated code reviews on this repo. Configure here.

@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from f2a06ab to bab7cc6 Compare September 28, 2026 20:28
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from b2a0f2f to f3a0ea8 Compare September 28, 2026 20:28
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from bab7cc6 to c224cd1 Compare September 28, 2026 21:20
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from f3a0ea8 to 2b97a8c Compare September 28, 2026 21:20
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from c224cd1 to ed51797 Compare September 29, 2026 15:20
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch 2 times, most recently from 1cab12b to ada9b35 Compare September 29, 2026 15:25
@kinyoklion
kinyoklion added this pull request to stack #2061 September 30, 2026 17:08
@github-actions

Copy link
Copy Markdown
Contributor

@launchdarkly/js-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 27281 bytes
Compressed size limit: 29000
Uncompressed size: 132642 bytes

@github-actions

Copy link
Copy Markdown
Contributor

@launchdarkly/js-client-sdk size report
This is the brotli compressed size of the ESM build.
Compressed size: 32691 bytes
Compressed size limit: 34000
Uncompressed size: 117095 bytes

@github-actions

Copy link
Copy Markdown
Contributor

@launchdarkly/js-client-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 25437 bytes
Compressed size limit: 44000
Uncompressed size: 165420 bytes

@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from ed51797 to 1f0e0c9 Compare September 30, 2026 18:17
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from ada9b35 to 7c91764 Compare September 30, 2026 18:17
@kinyoklion
kinyoklion removed this pull request from stack #2061 September 30, 2026 18:37
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 1f0e0c9 to ee3c511 Compare September 30, 2026 20:16
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 7c91764 to 5d8aa0c Compare September 30, 2026 20:16
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from ee3c511 to e4656e8 Compare September 30, 2026 20:23
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 5d8aa0c to 60f7458 Compare September 30, 2026 20:24
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from e4656e8 to cd8301d Compare September 30, 2026 21:01
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 60f7458 to 9765955 Compare September 30, 2026 21:01
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from cd8301d to f8bd9a5 Compare September 30, 2026 21:04
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 9765955 to c720aed Compare September 30, 2026 21:04
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from f8bd9a5 to 3a6273a Compare September 30, 2026 22:23
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from c720aed to bb5486f Compare September 30, 2026 22:23
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 3a6273a to 5095497 Compare September 30, 2026 22:27
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from bb5486f to e630c23 Compare September 30, 2026 22:27
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 5095497 to 7fdfeaf Compare October 1, 2026 23:07
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from e630c23 to 3fdf2c8 Compare October 1, 2026 23:08
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 7fdfeaf to 1fe976f Compare October 1, 2026 23:52
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 3fdf2c8 to f12acb8 Compare October 1, 2026 23:52
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 035f6ce to 2ba7a4a Compare October 8, 2026 22:35
@kinyoklion
kinyoklion marked this pull request as ready for review October 8, 2026 22:50
@kinyoklion
kinyoklion requested a review from a team as a code owner October 8, 2026 22:50
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 95d102a to 477a22f Compare October 9, 2026 18:50
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 2ba7a4a to a386f44 Compare October 9, 2026 18:50

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a386f44. Configure here.

@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 477a22f to f88c789 Compare October 9, 2026 19:58
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from a386f44 to c0f66ed Compare October 9, 2026 19:58
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from f88c789 to 40fb163 Compare October 9, 2026 20:30
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from c0f66ed to ab186ce Compare October 9, 2026 20:30
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 40fb163 to e596425 Compare October 9, 2026 21:02
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from ab186ce to e368a7c Compare October 9, 2026 21:02
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from e596425 to 29eea7c Compare October 9, 2026 21:27
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from e368a7c to 8c591bd Compare October 9, 2026 21:27
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 29eea7c to 2e95aa3 Compare October 9, 2026 22:08
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 8c591bd to 7a139be Compare October 9, 2026 22:08
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 2e95aa3 to 5202171 Compare October 9, 2026 22:41
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 7a139be to 712b6cc Compare October 9, 2026 22:41
Adds the file-based override source described by the OVERRIDE specification,
configured as { type: 'file', paths: [...] } in the overrides option of the
data system options.

The source reads one or more JSON or YAML files in the file data source
document format and combines their entries in the configured order. Duplicate
keys across files fail the reload by default or keep the first file's entry
with the ignore handling. Change detection is one of two modes: polling, the
default, examines the files once per second with a one second minimum, and
watching reacts to change notifications for the files' directories. A
configured file that does not exist contributes no overrides, so a file can be
created later and deleting a file removes its overrides. A file that exists
but cannot be read or parsed fails that reload, keeps the last good overrides,
logs the failure, and retries. Every applied change is logged at Info level
with the overrides in effect and what each file supplied. The initial load
completes before the client evaluates anything.

An invalid configuration, such as no file paths or an unrecognized change
detection mode, fails client construction. An unrecognized duplicate keys
handling and a polling interval below the minimum are logged and replaced by
their defaults.

The Node.js SDK reads YAML files without configuration through the yaml
package, which it supplies to the shared code as a platform default. A
configured yamlParser takes precedence.
… source

Evaluation no longer waits for the override source's initial load, so the
tests that evaluated right after creating the client now wait until the
override shows, as the tests of later changes already did.
…ile override source

The override layer now rejects a definition that evaluation cannot read.
For the file-based source that is a failed load like any other: the error
names the entry and the field, the last good overrides stay in effect, the
retry repeats at debug level, and a fix recovers.
…tests

The file override source expanded a flag value with its own copy of makeFlagWithValue; it now calls the file data source's with version 1, as the FDv2 file initializer does. The document parser shares the plain-object check with the definition validator and uses the common isNullish. The tests share one waitFor helper per package and the jest mock logger instead of per-file copies.

The server-node override tests asserted on an evaluation made right after the client was constructed. Evaluation does not wait for the initial override load, so those assertions raced the file read and failed on a busy machine. They now wait for the override to be in effect before asserting, like the rest of the file's tests.
The file override source merged and de-duplicated entries by the map key under which a file listed them, but the layer stored each entry under its key field, which the parser kept when the file supplied one. A definition pasted under a new map key with its old key field then overrode the old flag, escaped the duplicate check in the default fail mode, and inverted the ignore mode. The parser now sets the key field to the map key, as the Go and Python SDKs key the layer by the map key.
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-events branch from 5202171 to 9b26b7e Compare October 9, 2026 23:18
@kinyoklion
kinyoklion force-pushed the rlamb/overrides-node-file-source branch from 712b6cc to 0329a1f Compare October 9, 2026 23:18

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant