docs: add vector, geospatial, and full-text search guide#4
docs: add vector, geospatial, and full-text search guide#4richardsimmonds wants to merge 1 commit into
Conversation
New docs/search-guide.md covers all three DocumentDB search capabilities with index creation examples, sample queries, parameter references, and pymongo/mongosh code snippets. - Vector search: HNSW and IVF index creation; $search/cosmosSearch and $vectorSearch aggregation stages; similarity metrics (COS, L2, IP); efSearch/nProbes tuning; filtered and exact search; score projection - Geospatial search: 2d and 2dsphere index creation; $geoNear aggregation stage with spherical/Cartesian distance; $geoWithin query operator with $box, $geometry, $center, $centerSphere shapes; parameter reference - Full-text search: text index creation (single, compound, wildcard); $text operator with phrase, negation, and OR syntax; language/stemming support; textScore projection; parameter reference Also adds search-guide link to README Helpful Links section alongside the existing architecture and build-from-source references. Signed-off-by: richardsimmonds <richardsimmonds314@gmail.com>
Docs Review — NEEDS CHANGESOverall this is a solid first pass. The structure is logical, the examples are readable, and the coverage is broad. The cross-check against source ( Blocking issues (must fix before merge)
Blocking issues in README
Non-blocking notes
Review by docs-reviewer (Hermes Agent) |
richardsimmonds
left a comment
richardsimmonds
left a comment
There was a problem hiding this comment.
Inline comments attached — see summary comment above for full verdict.
| | **Query time** | Faster, better recall | Slightly slower, tunable | | ||
| | **Memory** | Higher | Lower | | ||
| | **Tune at query time** | `efSearch` | `nProbes` | | ||
| | **Max dimensions** | 2000 | 2000 | |
There was a problem hiding this comment.
Accuracy error. Max dimensions shows 2000 for both HNSW and IVF, but 2000 is only the non-compressed limit. Source: VECTOR_MAX_DIMENSIONS = 16000, VECTOR_MAX_DIMENSIONS_NON_COMPRESSED = 2000 in pg_documentdb/include/vector/vector_common.h. Consider: '2000 (uncompressed; up to 4000 with half-compression, 16000 with pq)' — or add a 'compression' row. Either way the current 2000 is misleading for anyone using large embedding models.
| vector: queryEmbedding, | ||
| path: "embedding", | ||
| k: 5, | ||
| efSearch: 40 // higher = better recall, slower (default: 2*k, min: 1) |
There was a problem hiding this comment.
Accuracy error. Comment says default: 2*k. The actual constant is HNSW_DEFAULT_EF_SEARCH = 40 (pg_documentdb/include/vector/vector_common.h:42). Change to // higher = better recall, slower (default: 40, min: 1).
| vector: queryEmbedding, | ||
| path: "visual_embedding", | ||
| k: 10, | ||
| nProbes: 20 // higher = better recall, slower (default: numLists, min: 1) |
There was a problem hiding this comment.
Accuracy error. Comment says default: numLists. The actual default is dynamically calculated in CalculateIVFSearchParamBson() (vector_index_kind_impl.c): for collections < 10,000 rows the default is numLists; for larger collections it's ceil(10000 / rowsPerCluster) capped at numLists. Suggest: // omit to let DocumentDB choose based on collection size (default: 1 when index metadata unavailable) — or just drop the default note and point readers to the parameter table.
| | `dimensions` | Yes | int | — | 1–2000 | Vector length | | ||
| | `m` | No | int | 16 | 2–100 | Max connections per HNSW layer. Higher = better recall, more memory | | ||
| | `efConstruction` | No | int | 64 | must be >= 2*m | Build-time beam width. Higher = better recall at build time | | ||
| | At query time: `efSearch` | No | int | 2*k | >= 1 | Search beam width. Higher = better recall at query time | |
There was a problem hiding this comment.
Accuracy error. Parameter table says default 2*k. Should be 40 — matches HNSW_DEFAULT_EF_SEARCH constant in vector_common.h.
| | `similarity` | Yes | string | — | `"COS"`, `"L2"`, `"IP"` | Distance metric | | ||
| | `dimensions` | Yes | int | — | 1–2000 | Vector length | | ||
| | `numLists` | No | int | 100 | >= 1 | Number of IVF clusters. Aim for sqrt(N) | | ||
| | At query time: `nProbes` | No | int | `numLists` | >= 1 | Clusters to probe. Higher = better recall | |
There was a problem hiding this comment.
Accuracy error. Parameter table says default numLists. The actual default is dynamically calculated (see inline comment at line 216). Remove the static value or replace with 'dynamic; see query-time tuning note'.
| | Parameter | Default | Description | | ||
| |-----------|---------|-------------| | ||
| | `default_language` | `"en"` | Language for stemming and stop-word filtering | | ||
| | `textIndexVersion` | 3 | Text index version. Use 3 (current) | |
There was a problem hiding this comment.
Accuracy error — will cause runtime failure. Default is 3, guidance says 'Use 3 (current)'. The code at create_indexes.c:1647 sets the internal default to 2, and line 4306 throws COMMANDNOTSUPPORTED for any value other than 2. If a user follows this doc and passes textIndexVersion: 3, their createIndex call will fail. Fix: change default to 2, guidance to 'Only version 2 is currently supported'.
| - Check out our [website](https://documentdb.io) to stay up to date with the latest on the project. | ||
| - Contributors and users can join the [DocumentDB Discord channel](https://discord.gg/vH7bYu524D) for quick collaboration. | ||
| - Check out [FerretDB](https://github.com/FerretDB/FerretDB) and their integration of DocumentDB as a backend engine. | ||
| - Want to build from source, run tests locally, or hack on the extensions? See [docs/build-from-source.md](docs/build-from-source.md). |
There was a problem hiding this comment.
Broken link. docs/build-from-source.md does not exist in this repo (not in main, not in this branch). Will 404 on merge. Remove this line from this PR, or add a stub file.
| - Contributors and users can join the [DocumentDB Discord channel](https://discord.gg/vH7bYu524D) for quick collaboration. | ||
| - Check out [FerretDB](https://github.com/FerretDB/FerretDB) and their integration of DocumentDB as a backend engine. | ||
| - Want to build from source, run tests locally, or hack on the extensions? See [docs/build-from-source.md](docs/build-from-source.md). | ||
| - Curious how the components fit together? See [docs/architecture.md](docs/architecture.md) for the data flow, wire protocol translation, and BSON type system overview. |
There was a problem hiding this comment.
Broken link. docs/architecture.md does not exist in this repo (not in main, not in this branch). Same issue — remove or stub.
1 participant
Summary
Adds
docs/search-guide.md— a comprehensive guide to DocumentDB's three search capabilities, which were previously advertised with zero documentation or examples.Closes the docs gap for vector search, geospatial search, and full-text search (kanban task t_20b995f9).
What's covered
Vector search
cosmosSearchOptions$search/cosmosSearchand Atlas-compatible$vectorSearchaggregation stagesefSearch/nProbesquery-time tuning, filtered search, exact brute-force searchGeospatial search
2dand2dsphereindex creation, GeoJSON and legacy coordinate storage$geoNearaggregation stage (spherical + Cartesian),$geoWithinwith$box/$geometry/$center/$centerSphereFull-text search
$textoperator: phrase, negation, OR syntax; language/stemming support;textScoreprojectionAlso adds a search-guide link to the README Helpful Links section, and a Combining Search Types + Index Management section.
Verification
Every operator, kind string, parameter name, and similarity metric was cross-checked against the source in
pg_documentdb/src(create_indexes.c, bson_aggregation_vector_search.c, opclass/, etc.). No invented features.Notes