fix(stream-parser): respect backpressure when importing readable streams - #724
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The change is small, aligns with Node stream backpressure semantics, preserves the documented fallback behavior, and is covered by targeted regression tests.
Pull request overview
This PR updates StreamParser#import() to respect Node stream backpressure when the imported source is a proper readable stream, preventing the parser from buffering the entire input when its output is not being consumed.
Changes:
- Switch
StreamParser#import()to usestream.pipe(this)when available, preserving backpressure behavior. - Keep explicit source error forwarding and retain the existing
data/endlistener fallback for non-stream event sources. - Add regression tests covering (1) backpressure pausing/resuming and (2) the non-
pipe()EventEmitter fallback behavior.
File summaries
| File | Description |
|---|---|
| test/N3StreamParser-test.js | Adds regression tests for backpressure behavior and for importing an EventEmitter-only source. |
| src/N3StreamParser.js | Uses pipe() for readable streams to respect backpressure; preserves error forwarding and fallback event-based import. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Contributor
|
🎉 This PR is included in version 2.7.6 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use
stream.pipe(this)when importing a readable stream so its producer pauses when the parser's buffers fill and resumes as quads are consumed. Previouslyimport()attached adatalistener and ignoredwrite()returning false, allowing an unread parser to consume and retain the entire source.Keep explicit source-error forwarding and the existing data/end listener fallback for RDF/JS event sources that do not provide
pipe(). The return value remains the parser.The regression test imports 30,000 one-triple chunks, checks that production pauses before exhausting the input, then consumes and verifies all 30,000 quads. It fails on the original implementation and passes with this patch. A second test covers an EventEmitter-only source.
Validation: all 6,891 tests pass with 100% coverage; ESLint, Node/browser builds, and
git diff --checkpass.This fixes pre-existing behavior on
mainand is independent of #721 and the separate readable-buffer initialization change.