feat(parquet): prepare range and worker foundations - #3539
Merged
Conversation
ibgreen
marked this pull request as ready for review
August 3, 2026 01:29
ibgreen
force-pushed
the
codex/parquet-source-prep
branch
from
August 3, 2026 01:34
ea1c0e9 to
a154fc1
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ea1c0e9053
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
ibgreen
force-pushed
the
codex/parquet-source-prep
branch
2 times, most recently
from
August 3, 2026 01:44
e5d9e53 to
5de2429
Compare
ibgreen
force-pushed
the
codex/parquet-source-prep
branch
from
August 3, 2026 13:48
5de2429 to
2cf901d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goals
ParquetSourcecapabilities, cached state, cancellation, and package-local WASM delivery explicit.Changes
HttpFilearound exact validated byte ranges, pinned ETag/Last-Modified identity, strict or best-effort consistency, custom fetch options, cancellation, and immutable transport telemetry.HttpFilelifecycle/scheduler façade from the focused HTTP transport, response-validation, telemetry, and public-type modules.416response withContent-Range: bytes */0, preserving validators and zero-length reads.RangeRequestSchedulerwith explicit transport-context isolation, exact response validation, safe EOF clamping through authoritative source length, prompt per-caller cancellation, and network timing.ParquetSourcecapability reporting that reflects the current range-backed TypeScript source: statistics, custom transport, object validation, and network/decode telemetry are supported; source worker decoding remains deferred.AbortSignal.reason.@loaders.gl/parquet/wasmURL helper and raw WASM asset export, and updatesparquet-wasmto 0.7.2.HttpFilecoverage into the shared test project and verifies it in Node and Chromium.HttpFileAPI reference covering opening, options, consistency modes, response requirements, methods, telemetry, cancellation, CORS, and scheduler sharing.ReadableFileexamples that used a nonexistentslice()method.Current boundary
ParquetSourcecontinues to use the selective TypeScript range backend merged in #3531, #3533, and #3535. It does not introduce the obsolete whole-file WASM source/session alternative from the pre-stack prototype. Worker-backed source decoding remains advertised as unavailable; the canonical WASMParquetLoaderworker path is already provided by #3538.Validation
yarn installyarn lint fixyarn buildcd website && yarn buildHttpFilesuite — 14 tests passed in Node and 14 in Chromiumyarn test-node— 403 files passed, 3 skipped; 1,915 tests passed, 81 skippedyarn test-headless— 406 files passed, 3 skipped; 1,881 tests passed, 60 skippedtestworkflow — 7 jobs passedgit diff --checkStack
master(fix(parquet): keep source runtime off package root #3530, feat(parquet): stream selective source batches #3531, feat(parquet): close TypeScript decoder gaps #3533, feat(parquet): add statistics, pruning, and telemetry #3535, and feat(parquet): add cancellable worker decode #3538 are merged)Part of #3525.