Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
- Never abbreviate variables, always type out the full name in camelCase (variables, functions, fields), PascalCase (types), CAPITAL_CASE (constant)
- Add TSDoc to all new classes, functions, methods, fields.
- Prefer verbNoun structure for function and method names.
- Prefer `readonly` fields over getters for fixed or constructor-derived values.
- We end JavaScript statements with semicolons. Do not remove semicolons.

## Notes
Expand Down
26 changes: 26 additions & 0 deletions docs/modules/compression/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,32 @@
The `@loaders.gl/compression` module provides a selection of lossless,
compression/decompression "transforms" with a unified interface that work both in browsers and in Node.js

For async code that only needs decompression, the lightweight
[`@loaders.gl/compression/native-decompression`](/docs/modules/compression/api-reference/native-decompression)
entrypoint probes the runtime's
`DecompressionStream` implementation for gzip, deflate, raw deflate, Brotli, and Zstandard. The
entrypoint has no codec imports, so supported runtimes do not pull fallback codec code into the
initial bundle. It returns `null` when the runtime or exact format is unavailable, allowing callers
to load a fallback only when needed.
<img src="https://img.shields.io/badge/From-v5.0-blue.svg?style=flat-square" alt="From-v5.0" />

```typescript
import {decompressWithNativeDecompressionStream} from '@loaders.gl/compression/native-decompression';

async function decompressGzip(input: ArrayBuffer): Promise<ArrayBuffer> {
const output = await decompressWithNativeDecompressionStream(input, 'gzip');
if (output) {
return output;
}
const {GZipCompression} = await import('@loaders.gl/compression/gzip-compression');
return new GZipCompression().decompress(input);
}
```

Parquet and SPZ parsing use this lightweight path automatically before lazily loading their
codec-backed fallbacks. Existing compression classes keep their deterministic codec behavior for
compression and synchronous decompression.

## API

| Compression Class | Format | Characteristics | Library Size | Notes |
Expand Down
2 changes: 2 additions & 0 deletions docs/modules/compression/api-reference/brotli-compression.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,5 @@ Implements the [`Compression](./compression) API.
## Methods

### `constructor(options?: object)`

`options` is optional when using the built-in Brotli decoder.
38 changes: 38 additions & 0 deletions docs/modules/compression/api-reference/native-decompression.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Native Decompression

The lightweight `@loaders.gl/compression/native-decompression` entrypoint exposes async
decompression through the runtime's `DecompressionStream` API without importing fallback codecs.
It supports `gzip`, `deflate`, `deflate-raw`, `brotli`, and forward-compatible `zstd`
constructor probing.
<img src="https://img.shields.io/badge/From-v5.0-blue.svg?style=flat-square" alt="From-v5.0" />

The helpers return `null` only when `DecompressionStream` or the requested format is unavailable.
After a native stream is created, decompression errors are propagated to the caller.

```typescript
import {
decompressWithNativeDecompressionStream
} from '@loaders.gl/compression/native-decompression';

async function decompressGzip(compressedData: ArrayBuffer): Promise<ArrayBuffer> {
const output = await decompressWithNativeDecompressionStream(compressedData, 'gzip');
if (output) {
return output;
}
const {GZipCompression} = await import('@loaders.gl/compression/gzip-compression');
return new GZipCompression().decompress(compressedData);
}
```

## Functions

### `decompressWithNativeDecompressionStream(input, format)`

Decompresses one `ArrayBuffer` and returns an exact `ArrayBuffer`, or `null` when the runtime
does not support the requested format.

### `decompressBatchesWithNativeDecompressionStream(inputBatches, format)`

Creates an incremental native decompression stream for iterable or async iterable `ArrayBuffer`
batches. It returns an async iterable of exact `ArrayBuffer` chunks, or `null` when the runtime
does not support the requested format.
8 changes: 8 additions & 0 deletions docs/modules/compression/api-reference/zstd-compression.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,18 @@

Compresses / decompresses Zstandard encoded data.

Inject `zstd-codec` through `options.modules` for compression and decompression through this
codec-backed class. Async-only callers can probe future native Zstandard support through the
lightweight `@loaders.gl/compression/native-decompression` entrypoint without importing
`zstd-codec`.

## Interface

Implements the [`Compression](./compression) API.

## Methods

### `constructor(options?: object)`

`options` is optional at construction time. Supply `{modules: {'zstd-codec': ZstdCodec}}` before
calling compression or decompression methods.
8 changes: 6 additions & 2 deletions docs/modules/parquet/api-reference/parquet-loader.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,12 @@ field-level GeoArrow metadata.

## Compressions

Some compressions are big and need to be imported explicitly by the application
and passed to the `ParquetLoader`
Async Parquet parsing first probes the runtime's native `DecompressionStream` for gzip, Brotli,
and Zstandard pages. Those probes come from a lightweight entrypoint with no codec imports, and
codec-backed implementations are loaded only when native support is unavailable. Native Zstandard
support is not yet widely available, so inject `zstd-codec` for broad compatibility; when
provided, it takes precedence over the native path. LZ4 still requires `lz4js`.
<img src="https://img.shields.io/badge/From-v5.0-blue.svg?style=flat-square" alt="From-v5.0" />

```typescript
import {ParquetLoader} from '@loaders.gl/parquet';
Expand Down
10 changes: 8 additions & 2 deletions docs/modules/splats/api-reference/spz-loader.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,15 @@

## Usage

SPZ version 4 uses ZSTD-compressed attribute streams. Inject `zstd-codec` through loader options so applications that only use `SPLATLoader` or `KSPLATLoader` do not pay the ZSTD dependency cost.
SPZ version 4 uses ZSTD-compressed attribute streams. Async SPZ parsing first probes the lightweight
native decompression entrypoint, which has no codec imports, before lazily loading the
codec-backed fallback. Native Zstandard support is not yet widely available, so inject
`zstd-codec` through loader options for broad runtime compatibility; when provided, it takes
precedence over the native path.
<img src="https://img.shields.io/badge/From-v5.0-blue.svg?style=flat-square" alt="From-v5.0" />

```typescript
// Install zstd-codec for broad runtime compatibility.
// npm install @loaders.gl/core @loaders.gl/splats zstd-codec

import {load} from '@loaders.gl/core';
Expand Down Expand Up @@ -72,4 +78,4 @@ Schema metadata includes `loaders_gl.semantic_type = gaussian-splats` and `loade
| Option | Type | Default | Description |
| -------------- | --------------- | --------------- | ---------------------------------------- |
| `splats.shape` | `'arrow-table'` | `'arrow-table'` | Selects Mesh Arrow table output. V1 only supports `arrow-table`. |
| `modules` | `object` | `{}` | Must include `{'zstd-codec': ZstdCodec}` to decode SPZ version 4 streams. |
| `modules` | `object` | `{}` | Include `{'zstd-codec': ZstdCodec}` for broad SPZ version 4 runtime compatibility. |
4 changes: 4 additions & 0 deletions docs/whats-new.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ Release Date: 2026
- `preload(loader)` now returns and caches parser-bearing implementations for unbundled loaders.
- `preloadSync(loader)` NEW returns a cached parser-bearing implementation, or `null` if the loader has not been preloaded.

**@loaders.gl/compression**

- The new lightweight `@loaders.gl/compression/native-decompression` entrypoint probes runtime support for gzip, deflate, raw deflate, Brotli, and future Zstandard decompression without importing codec fallbacks. Parquet and SPZ use it before lazily loading codec-backed implementations; until native Zstandard support becomes widely available, most runtimes still need `zstd-codec` for Zstandard-backed parsing.

**@loaders.gl/deck-layers**

- `SplatLayer` NEW - renders GraphDECO-style Gaussian splat Arrow tables from `PLYLoader` or `@loaders.gl/splats`.
Expand Down
39 changes: 33 additions & 6 deletions modules/compression/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,38 @@
"import": "./dist/index.js",
"require": "./dist/index.cjs"
},
"./native-decompression": {
"types": "./dist/native-decompression.d.ts",
"import": "./dist/native-decompression.js"
},
"./no-compression": {
"types": "./dist/no-compression.d.ts",
"import": "./dist/no-compression.js"
},
"./gzip-compression": {
"types": "./dist/gzip-compression.d.ts",
"import": "./dist/gzip-compression.js"
},
"./deflate-compression": {
"types": "./dist/deflate-compression.d.ts",
"import": "./dist/deflate-compression.js"
},
"./brotli-compression": {
"types": "./dist/brotli-compression.d.ts",
"import": "./dist/brotli-compression.js"
},
"./snappy-compression": {
"types": "./dist/snappy-compression.d.ts",
"import": "./dist/snappy-compression.js"
},
"./lz4-compression": {
"types": "./dist/lz4-compression.d.ts",
"import": "./dist/lz4-compression.js"
},
"./zstd-compression": {
"types": "./dist/zstd-compression.d.ts",
"import": "./dist/zstd-compression.js"
},
"./compression-worker.js": {
"import": "./dist/compression-worker.js"
},
Expand Down Expand Up @@ -63,13 +95,8 @@
"pako": "1.0.11",
"snappyjs": "^0.6.1"
},
"optionalDependencies": {
"@types/brotli": "^1.3.0",
"brotli": "^1.3.2",
"lz4js": "^0.2.0",
"zstd-codec": "^0.1"
},
"devDependencies": {
"@types/brotli": "^1.3.0",
"brotli": "^1.3.2",
"lz4js": "^0.2.0",
"zstd-codec": "^0.1"
Expand Down
6 changes: 6 additions & 0 deletions modules/compression/src/brotli-compression.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// loaders.gl
// SPDX-License-Identifier: MIT
// Copyright (c) vis.gl contributors

export {BrotliCompression} from './lib/brotli-compression';
export type {BrotliCompressionOptions} from './lib/brotli-compression';
6 changes: 6 additions & 0 deletions modules/compression/src/deflate-compression.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// loaders.gl
// SPDX-License-Identifier: MIT
// Copyright (c) vis.gl contributors

export {DeflateCompression} from './lib/deflate-compression';
export type {DeflateCompressionOptions} from './lib/deflate-compression';
6 changes: 6 additions & 0 deletions modules/compression/src/gzip-compression.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// loaders.gl
// SPDX-License-Identifier: MIT
// Copyright (c) vis.gl contributors

export {GZipCompression} from './lib/gzip-compression';
export type {GZipCompressionOptions} from './lib/gzip-compression';
2 changes: 1 addition & 1 deletion modules/compression/src/lib/brotli-compression.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ export class BrotliCompression extends Compression {
readonly isSupported = true;
readonly options: BrotliCompressionOptions;

constructor(options: BrotliCompressionOptions) {
constructor(options: BrotliCompressionOptions = {}) {
super(options);
this.options = options;
registerJSModules(options?.modules);
Expand Down
Loading
Loading