The tsvector type

[tamnd]

Summary

This adds the type tsvector of text search, as tsvector.c, tsvector_parser.c and tsvector_op.c have it. The functions that need a text search configuration, such as to_tsvector, are not in this change.

Changes

• rupg-types has the new Lexeme and Value::TsVector, a list of lexemes in order. Each position keeps the weight in the two high bits, as WordEntryPos does.
• tsvector_parse follows the states of gettoken_tsvector: quotes, escapes, positions, weights and the errors 42601 and 54000. The input sorts the lexemes, merges equal lexemes and keeps up to 256 positions for each lexeme.
• tsvector_deparse, tsvector_send and tsvector_recv give the text and binary forms. The receive function has the checks of tsvectorrecv. Recv::cstring reads a string up to a zero byte, as pq_getmsgstring does.
• tsvector_cmp gives the order of silly_cmp_tsvector: the size, the number of lexemes, then the lexemes, the positions and the weights.
• rupg-func has length, strip, setweight, ts_delete, tsvector_to_array, array_to_tsvector, ts_filter, unnest, the operator ||, tsvector_cmp and the comparison operators.
• A constant of type tsvector in a stored expression keeps the bytes of its binary form.

Known differences

• PostgreSQL gives the value of memcmp from tsvector_cmp when two lexemes are different. That value can change with the C library. Here it is the difference of the first two bytes that are not equal.

Tests

• The session test tsvector is the output of PostgreSQL 19 for the same script in a new database. It has the input and its errors, the output, the order, each function, the limits of the positions and of a lexeme, arrays, GROUP BY, DISTINCT, text and binary COPY and a query with parameters.
• Unit tests for the parser, the order and the binary form.
• fmt, clippy, the workspace tests and the checks of xtask pass on server1.