Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/bloom-union-n-metadata.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"distillate": patch
---

Fix `BloomFilter.union` reporting the wrong `bitsPerKey`/`rate` and breaking `equals`: the result now carries the operand's expected-key count instead of re-deriving it from `m/k` (membership was never affected). `BloomFilter.create` now rejects `n` above `2^32-1` with a `ParamError` instead of silently truncating it into the serialized frame.
32 changes: 16 additions & 16 deletions packages/distillate/docs/api/bloom/classes/BloomFilter.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ filter.has("bob"); // false (or a ~1% false positive)

> **new BloomFilter**(`__namedParameters`): `BloomFilter`

Defined in: [packages/distillate/src/bloom/bloom.ts:130](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L130)
Defined in: [packages/distillate/src/bloom/bloom.ts:128](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L128)

Constructs a filter from low-level [BloomParams](../interfaces/BloomParams.md). Prefer
[BloomFilter.create](#create) unless restoring a specific configuration.
Expand All @@ -49,7 +49,7 @@ Constructs a filter from low-level [BloomParams](../interfaces/BloomParams.md).

> **get** **bitsPerKey**(): `number`

Defined in: [packages/distillate/src/bloom/bloom.ts:165](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L165)
Defined in: [packages/distillate/src/bloom/bloom.ts:172](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L172)

Analytic design bits-per-key `m / n`.

Expand All @@ -65,7 +65,7 @@ Analytic design bits-per-key `m / n`.

> **get** **k**(): `number`

Defined in: [packages/distillate/src/bloom/bloom.ts:150](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L150)
Defined in: [packages/distillate/src/bloom/bloom.ts:157](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L157)

Number of hash probes per key.

Expand All @@ -81,7 +81,7 @@ Number of hash probes per key.

> **get** **length**(): `number`

Defined in: [packages/distillate/src/bloom/bloom.ts:160](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L160)
Defined in: [packages/distillate/src/bloom/bloom.ts:167](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L167)

Number of bits currently set.

Expand All @@ -97,7 +97,7 @@ Number of bits currently set.

> **get** **m**(): `number`

Defined in: [packages/distillate/src/bloom/bloom.ts:145](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L145)
Defined in: [packages/distillate/src/bloom/bloom.ts:152](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L152)

Number of bits in the filter.

Expand All @@ -113,7 +113,7 @@ Number of bits in the filter.

> **get** **seed**(): `number`

Defined in: [packages/distillate/src/bloom/bloom.ts:155](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L155)
Defined in: [packages/distillate/src/bloom/bloom.ts:162](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L162)

Hash seed.

Expand All @@ -127,7 +127,7 @@ Hash seed.

> **add**(`key`): `void`

Defined in: [packages/distillate/src/bloom/bloom.ts:262](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L262)
Defined in: [packages/distillate/src/bloom/bloom.ts:272](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L272)

Adds a key to the set.

Expand All @@ -149,7 +149,7 @@ The key to insert, as a string or bytes.

> **equals**(`other`): `boolean`

Defined in: [packages/distillate/src/bloom/bloom.ts:207](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L207)
Defined in: [packages/distillate/src/bloom/bloom.ts:214](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L214)

Tests structural equality: `true` when `other` serializes to identical
bytes, meaning identical parameters and set bits.
Expand All @@ -174,7 +174,7 @@ The filter to compare against.

> **has**(`key`): `boolean`

Defined in: [packages/distillate/src/bloom/bloom.ts:273](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L273)
Defined in: [packages/distillate/src/bloom/bloom.ts:283](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L283)

Tests whether a key is in the set.

Expand All @@ -198,7 +198,7 @@ The key to test.

> **rate**(): `number`

Defined in: [packages/distillate/src/bloom/bloom.ts:176](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L176)
Defined in: [packages/distillate/src/bloom/bloom.ts:183](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L183)

Estimates the current false-positive rate from the actual fill,
`(length / m) ** k`. This reflects how full the filter is right now, not
Expand All @@ -216,7 +216,7 @@ The estimated false-positive rate, `0` for an empty filter.

> **toBytes**(): `Uint8Array`

Defined in: [packages/distillate/src/bloom/bloom.ts:185](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L185)
Defined in: [packages/distillate/src/bloom/bloom.ts:192](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L192)

Serializes the filter to a portable little-endian byte layout.

Expand All @@ -232,7 +232,7 @@ The serialized filter, readable by [BloomFilter.fromBytes](#frombytes).

> **toJSON**(): [`FilterJSON`](../interfaces/FilterJSON.md)

Defined in: [packages/distillate/src/bloom/bloom.ts:217](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L217)
Defined in: [packages/distillate/src/bloom/bloom.ts:224](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L224)

Serializes the filter to a JSON-friendly envelope wrapping the base64 of
[BloomFilter.toBytes](#tobytes).
Expand All @@ -249,7 +249,7 @@ The envelope, readable by [BloomFilter.fromJSON](#fromjson).

> **union**(`other`): `BloomFilter`

Defined in: [packages/distillate/src/bloom/bloom.ts:238](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L238)
Defined in: [packages/distillate/src/bloom/bloom.ts:245](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L245)

Returns a new filter containing the union of this filter and `other`.

Expand Down Expand Up @@ -307,7 +307,7 @@ A new, empty filter.

> `static` **from**(`keys`, `epsilon`): `BloomFilter`

Defined in: [packages/distillate/src/bloom/bloom.ts:88](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L88)
Defined in: [packages/distillate/src/bloom/bloom.ts:87](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L87)

Builds a filter from `keys`, sized for their count at the target
false-positive rate. The ergonomic entry point when the key set is already
Expand Down Expand Up @@ -339,7 +339,7 @@ A new filter containing every key.

> `static` **fromBytes**(`bytes`): `BloomFilter`

Defined in: [packages/distillate/src/bloom/bloom.ts:101](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L101)
Defined in: [packages/distillate/src/bloom/bloom.ts:100](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L100)

Restores a filter from its [BloomFilter.toBytes](#tobytes) serialization.

Expand All @@ -363,7 +363,7 @@ The reconstructed filter.

> `static` **fromJSON**(`value`): `BloomFilter`

Defined in: [packages/distillate/src/bloom/bloom.ts:227](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L227)
Defined in: [packages/distillate/src/bloom/bloom.ts:234](https://github.com/akshay-xp/distillate/blob/main/packages/distillate/src/bloom/bloom.ts#L234)

Restores a filter from its [BloomFilter.toJSON](#tojson) envelope.

Expand Down
22 changes: 16 additions & 6 deletions packages/distillate/src/bloom/bloom.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,9 @@ export class BloomFilter {
*/
static create(n: number, epsilon: number): BloomFilter {
assertPositiveInt(n, "n");
assertUint32(n, "n");
assertProbability(epsilon, "epsilon");
const f = new BloomFilter(optimal(n, epsilon));
f.#n = n;
return f;
return BloomFilter.#withN(optimal(n, epsilon), n);
}

/**
Expand Down Expand Up @@ -117,8 +116,7 @@ export class BloomFilter {
const seed = dv.getUint32(6, true);
const n = dv.getUint32(10, true);
assertBodyLength(body.length, 14 + Math.ceil(m / 8), "bloom");
const f = new BloomFilter({ m, k, seed });
f.#n = n;
const f = BloomFilter.#withN({ m, k, seed }, n);
f.#bits.bytes.set(body.subarray(14));
return f;
}
Expand All @@ -141,6 +139,15 @@ export class BloomFilter {
this.#n = Math.round((m * Math.LN2) / k);
}

// Reconstruct with an explicit expected-key count, overriding the #n the
// constructor derives from m/k. The single place #n is carried across
// reconstruction, so a caller cannot silently drop it.
static #withN(params: BloomParams, n: number): BloomFilter {
const f = new BloomFilter(params);
f.#n = n;
return f;
}

/** Number of bits in the filter. */
get m(): number {
return this.#m;
Expand Down Expand Up @@ -249,7 +256,10 @@ export class BloomFilter {
const b = other.#bits.bytes;
const merged = new Uint8Array(a.length);
for (let i = 0; i < a.length; i++) merged[i] = (a[i] ?? 0) | (b[i] ?? 0);
const r = new BloomFilter({ m: this.#m, k: this.#k, seed: this.#seed });
const r = BloomFilter.#withN(
{ m: this.#m, k: this.#k, seed: this.#seed },
this.#n,
);
r.#bits.bytes.set(merged);
return r;
}
Expand Down
13 changes: 13 additions & 0 deletions packages/distillate/tests/bloom/bloom.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ test("create rejects invalid n and epsilon", () => {
for (const epsilon of [1, 0, -0.1, NaN]) {
expect(() => BloomFilter.create(100, epsilon)).toThrow(ParamError);
}
expect(() => BloomFilter.create(2 ** 32, 0.999)).toThrow(ParamError);
expect(() => BloomFilter.create(2 ** 32 - 1, 0.999)).not.toThrow();

const f = BloomFilter.create(100000, 0.01);
f.add("alice");
expect(f.has("alice")).toBe(true);
Expand Down Expand Up @@ -198,6 +201,16 @@ test("union merges two filters without mutating inputs", () => {
expect(b.toBytes()).toEqual(snapB);
});

test("union preserves the operand bitsPerKey and equals the receiver", () => {
const a = BloomFilter.create(1000, 0.01);
a.add("x");
const b = BloomFilter.create(1000, 0.01);

const u = a.union(b);
expect(u.bitsPerKey).toBe(a.bitsPerKey);
expect(a.equals(u)).toBe(true);
});

test("union has every key from either input (property)", () => {
fc.assert(
fc.property(
Expand Down