The weft thread. The warp threads never touch each other; one weft crosses and binds them all.
A search engine where ranking signals are interchangeable. Go, from scratch, standard library only.
// Every scorer implements this. Fusion knows only this.
type Scorer interface {
Name() string
Candidates(ctx context.Context, q Query, k int) ([]Candidate, error)
}
// Knows neither how many scorers there are nor what any of them compute.
func Fuse(streams [][]Candidate, k int) []CandidateHybrid search engines started with one signal and bolted the rest on. Fusion ended up a special case: a dedicated code path joins two signals, and a third means rewriting it. That is why graph proximity is not a first-class ranking signal in any engine.
weft inverts the order. Fusion is the default operation, scorers plug into it, and the fourth scorer costs what the first did.
If you need text + vector hybrid search today, use bleve — it already has BM25, ANN and RRF. weft rests on an architectural hypothesis, not a market gap. docs/FINDINGS.md records how far that hypothesis is verified.
Milestones 1–7 and 15–23 are done or measured.
Not usable in production. A commit holds the write lock for as long as it takes: 11 seconds for a 20,000-document batch, with reads queued behind it. Sustained query load collapses at 27 queries per second rather than degrading.
One result is worth knowing before you read anything else: graph proximity does not improve ranking. At its best fusion weight it is worth +0.0000 nDCG@10. At equal weight it costs 0.1227. The signal is not harmful; it is not information. That is this project's own second falsification condition, coming back negative in public.
Per-milestone state, the published nDCG table, and the two debts milestone 5 has not paid: docs/STATUS.md.
go run ./examples/breakdownquery "ranking fusion" @ [1 0 0], 4 scorers
1. rrf 0.04813 text:2 vector:3 graph:- recency:2
2. tfidf 0.04791 text:1 vector:2 graph:- recency:5
3. bm25 0.03366 text:- vector:1 graph:1 recency:4
4. hnsw 0.03311 text:- vector:4 graph:2 recency:3
5. changelog 0.01639 text:- vector:- graph:- recency:1
The trailing columns are each scorer's rank before fusion.
tfidfleads text but lands second. No other scorer put it first, and one scorer's confidence does not beat consensus.- A
-means the document is absent from that stream: no opinion, withheld as a traversal seed, or below the cut. changeloghas neither a vector nor a link and surfaces anyway. That is what rank fusion buys.
The example fuses with FuseWeighted(1, 1, 0.1, 1), discounting the graph stream to a tenth of a vote. That is the one weight this project has measured.
Three more examples, one case each:
| Example | Case |
|---|---|
examples/weights |
one query under Fuse and FuseWeighted, side by side |
examples/sparse |
the documents a scorer cannot see |
examples/basic |
the smallest embedding there is |
Godoc example: Example in pkg/engine.
go run ./cmd/weft index -data ./ix < corpus.jsonl
go run ./cmd/weft search -data ./ix -q '+fusion' -scorers vector,recency -breakdown
go run ./cmd/weft inspect -data ./ix -term fusionweft has five subcommands: index, search, inspect, check and encode. Between them they reach every callable symbol the library exports but three, and those three are named with their reason in cmd/weft/coverage_test.go. That test reads the golden API files as data, so an export no command can call fails the build.
Everything above assumes a clone and a Go toolchain. Two ways to the same binaries without either:
brew install skyoo2003/tap/weft # macOS and Linux; v0.1.0 installs weft alone, weftd from the next tag
docker run --rm -p 9200:9200 -v weft-data:/var/lib/weft ghcr.io/skyoo2003/weft:v0.1.0The image holds both binaries and runs weftd. The CLI is one flag away:
docker run --rm -i -v weft-data:/var/lib/weft --entrypoint weft \
ghcr.io/skyoo2003/weft:v0.1.0 index -data /var/lib/weft < corpus.jsonlThree things the image does that the commands above do not:
- It binds
0.0.0.0. The loopback defaultweftduses everywhere else would make a published port unreachable, so the image overrides it — which means what-ppublishes is open, and there is still no authentication and no TLS. - It runs as uid 65534. A named volume inherits that ownership and works as written; a bind mount does not, so pass
--user "$(id -u):$(id -g)"if you mount a host directory. - It has no
latesttag, and no movingv0.1either. The API may break inside a minor while this is v0.x, so the version you pin is the version you tested.
go run ./cmd/weftd # 127.0.0.1:9200
curl -XPUT localhost:9200/papers
curl -XPUT 'localhost:9200/papers/_doc/1?refresh=true' \
-H 'Content-Type: application/json' -d '{"text":"reciprocal rank fusion"}'
curl -XPOST localhost:9200/papers/_search \
-H 'Content-Type: application/json' -d '{"query":{"match":{"text":"fusion"}}}'weftd speaks a subset of the OpenSearch REST API, and opensearch-py drives it unmodified — make compat is that check.
GET / reports OpenSearch 2.19.0, which is untrue and is the only untrue thing it says. Past the handshake, a query weft cannot express returns 400 or 501 with a reason, never 200 with an empty hit list. DECISIONS D-025 and D-026 argue both halves.
The subset: match, match_phrase, term, terms, prefix, wildcard, fuzzy, range, exists, one level of bool, knn, hybrid, function_score decay, _bulk, and from/size paging. Six of twenty-five documented rows refuse — 24%, counted by a test rather than by eye. LIMITATIONS lists what each refusal is protecting you from.
The hybrid is the point of the server, not a feature of it:
curl -XPOST localhost:9200/papers/_search -H 'Content-Type: application/json' -d '{
"query": {"hybrid": {"queries": [
{"match": {"text": "rank fusion"}},
{"knn": {"vec": {"vector": [0.1, 0.9, 0.2], "k": 10}}},
{"function_score": {"gauss": {"published": {}}}}],
"weights": [1, 1, 0.5]}}}'Three signals, one request, no search pipeline and no normalization processor. The weights attach to positions, so fusion.FuseWeighted still cannot name a single scorer. That is the architecture claim, restated on the far side of a socket.
The library is still the product. The server is a cmd/: the DSL, the mappings, _bulk and the hybrid together changed zero lines under pkg/. Adding recency as a fourth signal changed zero lines of fusion code and 57 lines in its query clause. What it did cost is an index-time binding, because a JSON body has no field for "this date is the document's time" (FINDINGS milestone 26).
The production warning above applies unchanged, and weftd binds to loopback because there is no authentication and no TLS.
hybrid is a wrapper around a list of streams. On weft's own route the list is the request, and two things the OpenSearch response has nowhere to put come back with it:
curl -XPOST localhost:9200/papers/_weft/search -H 'Content-Type: application/json' -d '{
"streams": [{"match": {"text": "rank fusion"}},
{"knn": {"vec": {"vector": [0.1, 0.9, 0.2], "k": 10}}}],
"weights": [1, 0.5], "depth": 100, "breakdown": true}'breakdownis each stream's own rank before fusion — the-column above, per hit, withnullfor a stream that had no opinion.depthis the fusion depth. On_searchit is tangled withknn'sk, and a text-only query cannot set it at all.
A stream is a leaf clause in the spelling _search already accepts, compiled by the same function, so every clause and every refusal is shared rather than reimplemented. API is the reference.
cd grpc && go run ./cmd/weftg -data ../.weftd-data # 127.0.0.1:9201The same request in a second encoding, and the root module still has no dependencies. grpc/ is a nested module with a replace ../, the shape bench/ already uses to keep bleve out, so go list -m all prints one line here.
The service decides nothing: it converts protobuf to the same struct the HTTP handler builds and calls the same function. A field on one side without a counterpart on the other fails the build. grpc/README.md is why it is a module, and make compat-grpc is a stock grpcio client driving it with stubs generated from weft.proto itself.
Each document answers one question, and only that one.
| Question | Document |
|---|---|
| How far along is it, and what do the numbers say? | STATUS |
| What does it not do? | LIMITATIONS |
| How is the module shaped? | ARCHITECTURE |
| How do I plug my own signal in? | SCORERS |
| How do I ask it something OpenSearch cannot express? | API |
| What was actually measured, milestone by milestone? | FINDINGS |
| What was decided, and why is it expensive to reverse? | DECISIONS |
| What is on disk? | FORMAT |
| How were the quality numbers produced? | EVAL, DATASETS |
| How were the latency numbers produced? | PERF |
| Is the documentation enough to contribute from? | ADOPTION |
| What else is out there? | RESEARCH |
Governance and process:
| Question | Document |
|---|---|
| How to build, check and submit a change | CONTRIBUTING.md |
| Where to take a bug, a proposal or a question | SUPPORT.md |
| A vulnerability — not the issue tracker | SECURITY.md |
| Behavior in this repository | Code of Conduct |
| Who decides what, and what a test decides instead | GOVERNANCE.md |
| How a tag gets cut | RELEASE.md |
| What changed between versions | CHANGELOG.md |
make all is the gate: fmt, build, vet, test -race. CI runs that same target, so nothing beyond the Go toolchain is needed to run what judges you.
CONTRIBUTING.md has the rest — every make target, which assertions a test decides for you, and what a pull request should say.
Apache License 2.0. Third-party notices: NOTICE — there are none.