perf: optimize hot paths and use native Set methods for set operations

[Copilot]

• Optimize isOperator() - replace regex with charCode-based validation
• Optimize computeExpression() - use Set for SYSTEM_VARS, avoid string splits, use Object.keys
• Optimize resolve() - add fast path for 2-segment selectors
• Optimize normalize() - use Object.keys with indexed for-loop
• Optimize walk() - use indexOf+substring instead of split+slice+join
• Optimize hashObject() - use in-place sort and indexed for-loop
• Optimize typeOf() - use direct string comparisons instead of hash lookup
• Optimize isEqual() - replace .every() with indexed for-loops
• Optimize has() - replace .every() with indexed for-loop
• Optimize Context class - use literal object and cached OpType values
• Optimize Query.compile() - use Set for top-level ops, Object.keys instead of Object.entries
• Use new native Set methods for set operators
  • intersection(): native Set.intersection() fast path for primitive arrays
  • unique(): native Set fast path for primitive arrays
  • $setDifference: native Set.difference() fast path
  • $setIsSubset: native Set.isSubsetOf() fast path
  • $setEquals: native Set.isSubsetOf() + size comparison fast path
• Add inline comments explaining charCode ranges in isOperator()
• Run all tests - 1859 tests passing, no regressions
• CI: remove Node.js v18/v20, use v22+ (required for native Set methods)
• Internalize Set method types in src/set-methods.d.ts so consumers don't need esnext.collection in their tsconfig

---

⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.

[pubkey]

@copilot check if the new Set methods can help to improve the performance https://developer.mozilla.org/en-US/blog/javascript-set-methods/

[pubkey]

@copilot add commenta here to explaim what these numbers mwan

[Copilot]

Added inline comments on each charCode range in 6851025. Each condition now has a trailing comment: // a-z, // A-Z, // 0-9, // _.

[Copilot]

@copilot check if the new Set methods can help to improve the performance https://developer.mozilla.org/en-US/blog/javascript-set-methods/

Added native Set method fast paths in commit 5888189:

• intersection() → Set.intersection() for primitive arrays
• unique() → new Set() for primitive arrays
• $setDifference → Set.difference() for primitive arrays
• $setIsSubset → Set.isSubsetOf() for primitive arrays
• $setEquals → Set.isSubsetOf() + size check for primitive arrays

Uses the same dual-path pattern already established in the $in operator: check if all elements are primitives, use native Set methods if so, fall back to HashMap for complex objects. Added esnext.collection to tsconfig lib for type support. All 1859 tests pass.

[pubkey]

@copilot how much faster is this comparecto the baseline?

[Copilot]

@copilot how much faster is this comparecto the baseline?

Here are the benchmark results comparing before/after across key operations:

Benchmark	Before	After	Improvement
Complex $and/$or match (single doc, 100K iter)	130.68ms	69.39ms	-47%
Nested field query over 10K docs	87.35ms	28.31ms	-68%
$match + $group over 100K docs	477.34ms	325.74ms	-32%
$match + $group over 10K docs	197.70ms	143.36ms	-28%
query.test() reuse over 100K docs	241.32ms	180.58ms	-25%
Simple equality query instantiation	34.61ms	27.08ms	-22%
$match+$project+$group+$sort (10K)	319.50ms	266.38ms	-17%

The biggest wins come from the hot-path optimizations (charCode-based isOperator, indexOf+substring in resolve/walk, indexed for-loops replacing .every() callbacks). The native Set methods provide additional gains specifically for set operations on primitive-only arrays.

[pubkey]

@copilot what is the performace difference only from using the new Set methods?

[Copilot]

@copilot what is the performace difference only from using the new Set methods?

Ran a benchmark isolating the native Set methods vs the HashMap fallback (both go through the full Aggregator pipeline, so the difference is purely the Set method contribution):

100-element arrays (1000 iterations):

Operator	Native Set	HashMap	Speedup
$setIntersection	28ms	163ms	83% faster
$setDifference	19ms	126ms	85% faster
$setIsSubset	13ms	78ms	83% faster
$setEquals	20ms	108ms	82% faster
$setUnion	16ms	145ms	89% faster

1000-element arrays (1000 iterations):

Operator	Native Set	HashMap	Speedup
$setIntersection	88ms	1361ms	94% faster
$setDifference	84ms	1056ms	92% faster
$setIsSubset	52ms	497ms	90% faster
$setEquals	75ms	1005ms	93% faster
$setUnion	106ms	1216ms	91% faster

For primitive-only arrays (numbers, strings, booleans), native Set methods are ~5-15x faster than the HashMap fallback. The advantage grows with array size. For arrays containing objects/nested documents, it falls back to the existing HashMap path automatically.

[pubkey]

@copilot CI fails because node v20 does not have the newest Set methods. Remove node v20 from the CI

[pubkey]

@copilot show me the different in the query matching performance with and without the new Set methods

[Copilot]

@copilot what is the performace difference only from using the new Set methods?

Ran a benchmark isolating the native Set methods vs the HashMap fallback (both go through the full Aggregator pipeline, so the difference is purely the Set method contribution):

100-element arrays (1000 iterations):

Operator	Native Set	HashMap	Speedup
$setIntersection	28ms	163ms	83% faster
$setDifference	19ms	126ms	85% faster
$setIsSubset	13ms	78ms	83% faster
$setEquals	20ms	108ms	82% faster
$setUnion	16ms	145ms	89% faster

1000-element arrays (1000 iterations):

Operator	Native Set	HashMap	Speedup
$setIntersection	88ms	1361ms	94% faster
$setDifference	84ms	1056ms	92% faster
$setIsSubset	52ms	497ms	90% faster
$setEquals	75ms	1005ms	93% faster
$setUnion	106ms	1216ms	91% faster

For primitive-only arrays (numbers, strings, booleans), native Set methods are ~5-15x faster than the HashMap fallback. The advantage grows with array size.

@copilot CI fails because node v20 does not have the newest Set methods. Remove node v20 from the CI

Removed Node.js v18 and v20 from the CI matrix, now only tests against v22+ which supports the native Set methods. See ec717cd.

[Copilot]

@copilot show me the different in the query matching performance with and without the new Set methods

Here are the benchmarks comparing native Set (primitives) vs HashMap fallback (objects) through the full Query/Aggregator pipeline:

Query matching — $in operator:

Benchmark	Native Set	HashMap	Speedup
$in (100 values, 1K docs, 500 iter)	2,159ms	17,767ms	88% faster
$in (500 values, 5K docs, 100 iter)	8,816ms	75,405ms	88% faster

Aggregation — Set expression operators (50-element arrays, 500 docs, 50 iter):

Operator	Native Set	HashMap	Speedup
$setIntersection	197ms	995ms	80%
$setDifference	176ms	803ms	78%
$setIsSubset	145ms	768ms	81%
$setEquals	143ms	653ms	78%
$setUnion	185ms	1,001ms	82%

Aggregation — Set expression operators (200-element arrays, 500 docs, 50 iter):

Operator	Native Set	HashMap	Speedup
$setIntersection	440ms	3,662ms	88%
$setDifference	491ms	2,890ms	83%
$setIsSubset	390ms	2,756ms	86%
$setEquals	385ms	2,301ms	83%
$setUnion	573ms	3,641ms	84%

Combined pipeline (intersection + difference + isSubset):

Benchmark	Native Set	HashMap	Speedup
arr=50, 2K docs, 20 iter	709ms	3,978ms	82%
arr=200, 2K docs, 20 iter	1,965ms	14,847ms	87%

Native Set methods are consistently ~5-8x faster for primitive arrays across both query matching and aggregation. The speedup increases with array size.

[pubkey]

@copilot do people who use this mingo library also have to add this to their compiler options? If yes, internalize the types so that they do not have to.

[Copilot]

No, consumers don't need it now. Removed esnext.collection from tsconfig.json lib and added a local ambient declaration file src/set-methods.d.ts that provides the Set method types internally. The .d.ts file is ambient so it doesn't get emitted in the build output. See edad842.