Repository navigation
feat: Add the file-based override source - #2053
kinyoklion wants to merge 9 commits into
Conversation
f2a06ab to
bab7cc6
Compare
b2a0f2f to
f3a0ea8
Compare
bab7cc6 to
c224cd1
Compare
f3a0ea8 to
2b97a8c
Compare
c224cd1 to
ed51797
Compare
1cab12b to
ada9b35
Compare
|
@launchdarkly/js-sdk-common size report |
|
@launchdarkly/js-client-sdk size report |
|
@launchdarkly/js-client-sdk-common size report |
ed51797 to
1f0e0c9
Compare
ada9b35 to
7c91764
Compare
1f0e0c9 to
ee3c511
Compare
7c91764 to
5d8aa0c
Compare
ee3c511 to
e4656e8
Compare
5d8aa0c to
60f7458
Compare
e4656e8 to
cd8301d
Compare
60f7458 to
9765955
Compare
cd8301d to
f8bd9a5
Compare
9765955 to
c720aed
Compare
f8bd9a5 to
3a6273a
Compare
c720aed to
bb5486f
Compare
3a6273a to
5095497
Compare
bb5486f to
e630c23
Compare
5095497 to
7fdfeaf
Compare
e630c23 to
3fdf2c8
Compare
7fdfeaf to
1fe976f
Compare
3fdf2c8 to
f12acb8
Compare
035f6ce to
2ba7a4a
Compare
95d102a to
477a22f
Compare
2ba7a4a to
a386f44
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a386f44. Configure here.
477a22f to
f88c789
Compare
a386f44 to
c0f66ed
Compare
f88c789 to
40fb163
Compare
c0f66ed to
ab186ce
Compare
40fb163 to
e596425
Compare
ab186ce to
e368a7c
Compare
e596425 to
29eea7c
Compare
e368a7c to
8c591bd
Compare
29eea7c to
2e95aa3
Compare
8c591bd to
7a139be
Compare
2e95aa3 to
5202171
Compare
7a139be to
712b6cc
Compare
Adds the file-based override source described by the OVERRIDE specification,
configured as { type: 'file', paths: [...] } in the overrides option of the
data system options.
The source reads one or more JSON or YAML files in the file data source
document format and combines their entries in the configured order. Duplicate
keys across files fail the reload by default or keep the first file's entry
with the ignore handling. Change detection is one of two modes: polling, the
default, examines the files once per second with a one second minimum, and
watching reacts to change notifications for the files' directories. A
configured file that does not exist contributes no overrides, so a file can be
created later and deleting a file removes its overrides. A file that exists
but cannot be read or parsed fails that reload, keeps the last good overrides,
logs the failure, and retries. Every applied change is logged at Info level
with the overrides in effect and what each file supplied. The initial load
completes before the client evaluates anything.
An invalid configuration, such as no file paths or an unrecognized change
detection mode, fails client construction. An unrecognized duplicate keys
handling and a polling interval below the minimum are logged and replaced by
their defaults.
The Node.js SDK reads YAML files without configuration through the yaml
package, which it supplies to the shared code as a platform default. A
configured yamlParser takes precedence.
…d use the default
… source Evaluation no longer waits for the override source's initial load, so the tests that evaluated right after creating the client now wait until the override shows, as the tests of later changes already did.
…ile override source The override layer now rejects a definition that evaluation cannot read. For the file-based source that is a failed load like any other: the error names the entry and the field, the last good overrides stay in effect, the retry repeats at debug level, and a fix recovers.
…tests The file override source expanded a flag value with its own copy of makeFlagWithValue; it now calls the file data source's with version 1, as the FDv2 file initializer does. The document parser shares the plain-object check with the definition validator and uses the common isNullish. The tests share one waitFor helper per package and the jest mock logger instead of per-file copies. The server-node override tests asserted on an evaluation made right after the client was constructed. Evaluation does not wait for the initial override load, so those assertions raced the file read and failed on a busy machine. They now wait for the override to be in effect before asserting, like the rest of the file's tests.
The file override source merged and de-duplicated entries by the map key under which a file listed them, but the layer stored each entry under its key field, which the parser kept when the file supplied one. A definition pasted under a new map key with its old key field then overrode the old flag, escaped the duplicate check in the default fail mode, and inverted the ignore mode. The parser now sets the key field to the map key, as the Go and Python SDKs key the layer by the map key.
5202171 to
9b26b7e
Compare
712b6cc to
0329a1f
Compare

Summary
This PR is based on the events branch because it completes the feature that the earlier branches build up, and it should merge after them.
This change adds the file-based override source described by the OVERRIDE specification to the Node.js server SDK. It is configured as
{ type: 'file', paths: [...] }in theoverridesproperty of the data system options, alongside the source object and factory forms that the earlier change added.The source reads one or more JSON or YAML files in the file data source document format, with optional
flags,flagValues, andsegmentsmembers, and combines their entries in the configured order. Duplicate keys across files fail the reload by default, or keep the first file's entry with theignorehandling. Change detection is one of two modes. Polling, the default, examines the files once per second, with a one second minimum, and works on every filesystem. Watching reacts to change notifications for the directories that contain the files. Change detection is in place before the initial load, so a change made between the two is not missed.A configured file that does not exist contributes no overrides, so a file can be created later and deleting a file removes its overrides. A file that exists but cannot be read or parsed fails that whole reload: the last good overrides stay in effect, the failure is logged once, and the source retries after one second. Every applied change is logged at Info level with the overrides in effect and what each file supplied, and an unchanged reload logs nothing. The initial load is part of starting the client: the promise that
waitForInitializationreturns settles after it, and an evaluation made before that reads the layer as it is at that moment.An invalid configuration fails client construction, the way an invalid file initializer does: no file paths, an unrecognized change detection mode, or a platform without filesystem support. An unrecognized duplicate keys handling is logged and replaced by its default. A polling interval below the minimum is logged and raised to one second, and one that is not a finite number or exceeds 2147483 seconds is logged and replaced by the default.
The specification requires YAML support without extra configuration. The Node.js SDK package gains a dependency on the
yamlpackage (2.9.1, already present in the lockfile as a transitive dependency of other workspaces) and supplies its parser to the shared code as a platform default. A configuredyamlParsertakes precedence. The shared server package has no new dependency.The existing file data sources keep their current behavior. The override source has its own document handling (format detection by file extension, validation of the document shape, entries keyed by their map key with a
keyfield inside an entry set to that map key, so a definition pasted under a new key overrides that key, value expansion to a flag that is on and serves the value by fallthrough, missing-file handling, retry) and none of it applies to the FDv1FileDataSourceor the FDv2 file data initializer.The tests mirror the Go file source tests over a mock filesystem: initial load, YAML, ordered merge, duplicate handling, missing files that appear and disappear, the Info log on each change, quiet watching of an absent file, reloads in both modes, retention across a malformed edit with recovery through the retry, and close. A server-node test drives the source through the client on real files, including the built-in YAML parser.
SDK-3247
Note
Overview
Adds an experimental file-based flag override source configurable via
dataSystem.overrides: { type: 'file', paths: [...] }. It loads JSON/YAML documents (same shape as file data sources:flags,flagValues,segments), merges multiple files in order, and hot-reloads via polling (default, 1s) or directory watching. Missing files contribute nothing; parse/duplicate-key failures keep the last good snapshot, log errors, and retry.Shared server code introduces
FileOverrideSource, override-specific document validation (parseOverrideDocument/fileOverrideSourcePolicy), and expandedcreateOverrideSourcevalidation (paths required, change-detection modes, poll interval bounds). Node server SDK adds theyamlpackage and registers a defaultyamlParseronLDClientNode; apps can override withyamlParserin override options.Evaluation behavior is unchanged except overrides sit on the read path:
overrideAffectedreasons,waitForInitializationwaits for the initial override load, and evaluations can use overrides before LD init when present in the layer.Reviewed by Cursor Bugbot for commit 0329a1f. Bugbot is set up for automated code reviews on this repo. Configure here.