This cookbook documents the public SQL foundation. Language compatibility and planned query vectors are tracked separately in the query feature maps, including the PromQL and LogsQL matrices. A similarly named SQL kernel is not by itself a claim of complete language semantics. Copyable statements mapped back to individual language rows are in SQL equivalents for query-language features.
Recipes for the query surface: the raw vtabs, the kernel TVFs
(timeless_aggregate, timeless_aggregate_frame, timeless_latest,
timeless_latest_frame, timeless_grid, timeless_window,
timeless_window_batches, timeless_raw_frame, timeless_rollup,
timeless_rollup_batches), bounded log row/count/value surfaces, bucket TVFs,
and catalog TVFs. The parameterized language mappings live in
QUERY_SQL_EQUIVALENTS.md and are executed by the
Rust query harness against the real extension so documentation drift fails
local validation.
Conventions used throughout:
tsis in the table's native unit (timeless_metrics= epoch seconds, generic logs = ms, traces = ns). The release logs server creates a microsecond table. The kernels are unit-agnostic:step,lookback, andwindoware in the same unit asts.- All kernel windows are half-open
(t − width, t]: a sample exactly attcounts, a sample exactly att − widthdoes not. - Results are sparse by default — grid points with no sample in the window produce no row. See Gap-fill to change that.
- Counter kernels are NOT PromQL: no extrapolation, no lookback
defaults, no staleness inference. Exact-percentile kernels are the
flip side: raw samples are kept, so
p95is exact, not anle-bucket estimate. - Binary metric batches and packed raw frames retain all IEEE value bits,
including distinct NaN payloads. Ordinary SQLite
REALprojection maps a NaN toNULL, and the kernels do not infer that every NaN is a Prometheus stale marker. SeePQL-S17in the feature matrix for the explicit deferred ingress/query prerequisites.
-- instant-selector shape: last sample per grid point, per series
SELECT labels, ts, value
FROM timeless_grid('metrics', 'cpu_usage', NULL, :t0, :t1, 60, 90);
-- range-vector shape: sliding-window op per grid point
-- folds sum|min|max|count|avg, counters delta|increase|rate,
-- exact percentiles pNN, trimmed mean tavg:N
SELECT labels, ts, value
FROM timeless_window('metrics', 'requests_total', NULL, :t0, :t1, 60, 300, 'rate');
-- the same kernel, packed into one versioned blob per series for embedded hosts
SELECT series_id, labels, buckets
FROM timeless_window_batches(
'metrics', 'requests_total', NULL, :t0, :t1, 60, 300, 'rate');
-- pre-aggregated tier read (declared via rollups='60@0' on the vtab)
SELECT labels, ts, value
FROM timeless_rollup('metrics', 'cpu_usage', NULL, 60, :t0, :t1, 'avg');
-- every rollup field in one versioned blob per matched series
SELECT series_id, labels, buckets
FROM timeless_rollup_batches(
'metrics', 'cpu_usage', NULL, 60, :t0, :t1);
-- one scalar reduction per matched series over inclusive bounds
SELECT series_id, labels, value
FROM timeless_aggregate('metrics', 'cpu_usage', NULL, :t0, :t1, 'avg');
-- the same scalar result for every series in one versioned frame
SELECT frame
FROM timeless_aggregate_frame(
'metrics', 'cpu_usage', NULL, :t0, :t1, 'avg');
-- newest point per matched series over inclusive bounds
SELECT series_id, labels, ts, value
FROM timeless_latest('metrics', 'cpu_usage', NULL, :t0, :t1);
-- every newest point in one versioned frame
SELECT frame
FROM timeless_latest_frame('metrics', 'cpu_usage', NULL, :t0, :t1);
-- every raw series in one versioned columnar frame for wide embedded reads
SELECT frame
FROM timeless_raw_frame(
'metrics', 'cpu_usage', NULL, :t0, :t1, :max_work_points);
-- label filters: plain string = equality; {"neq"|"re"|"nre": ...} match
-- against the whole value (anchored); absent label matches as ""
SELECT labels, ts, value FROM timeless_grid('metrics', 'cpu_usage',
'{"host": {"re": "web-.*"}, "env": {"neq": "dev"}}', :t0, :t1, 60, 90);
-- discovery: what metrics/series/labels exist? (no chunk reads)
SELECT * FROM timeless_series('metrics');
SELECT value FROM timeless_label_values('metrics', 'cpu_usage', 'host');
-- optional metric/matcher arguments filter before catalog rows cross SQLite
SELECT labels FROM timeless_series('metrics', 'cpu_usage',
'{"host": {"re": "web-.*"}, "env": {"neq": "dev"}}');
SELECT value FROM timeless_label_values('metrics', 'cpu_usage', 'host',
'{"env": {"neq": "dev"}}');
SELECT * FROM timeless_stats('metrics');
-- logs/traces frequency + latency dashboards
SELECT bucket_ts, group_key, n
FROM timeless_log_buckets('logs', 'level', NULL, :t0, :t1, 60000);
-- bounded ordered log rows: an output LIMIT does not replace the work guard
SELECT ts, level, message, metadata FROM logs
WHERE service='api' AND ts BETWEEN :t0 AND :t1
AND max_work_entries=:max_work_entries
ORDER BY ts DESC LIMIT 100 OFFSET 0;
-- exact count and bounded field discovery without rowset materialization
SELECT n FROM timeless_log_count(
'logs', '{"level":"error"}', NULL, :t0, :t1, :max_work_entries);
SELECT value FROM timeless_log_values(
'logs', 'host', '{"level":"error"}', NULL,
:t0, :t1, 1000, :max_work_entries);
SELECT bucket_ts, service, n, dur_p50, dur_p95, dur_p99
FROM timeless_trace_buckets('traces', NULL, :t0, :t1, 60000000000);
-- trace discovery from block metadata, including the live 8,192-span buffer
SELECT value FROM timeless_trace_services('traces');
SELECT value FROM timeless_trace_operations('traces', 'checkout');
-- bounded newest-first span search; LIMIT+OFFSET is pushed into the engine
SELECT * FROM traces
WHERE service = 'checkout' AND duration_ns >= 1000000
ORDER BY start_ts DESC, span_id DESC LIMIT 100 OFFSET 0;Unbounded timeless_traces scans stream one decoded block at a time. Inclusive
start_ts and duration_ns bounds plus exact service/kind/status/name filters
are applied inside the engine. When SQLite supplies an exact
ORDER BY start_ts[,span_id] ASC|DESC LIMIT/OFFSET shape, the engine retains
only LIMIT + OFFSET rows and stops at block timestamp bounds. Strict bounds
and unrecognized row predicates remain above the vtab and deliberately disable
bounded planning.
Trace service/operation discovery reads posting-list metadata rather than span payloads. New blocks carry a collision-free service/operation pair term; if a selected legacy block lacks the generation marker, operation discovery falls back to exact block-at-a-time decode. Upgrades therefore remain complete.
max_work_entries is an optional positive, inclusive guard over buffered
entries considered plus candidate persisted entries charged before payload
decode. It is independent of result cardinality: LIMIT 1 cannot conceal a
database-wide scan. Exceeding the cap returns an error and no partial rows,
count, or value set. A bound-pruned block is not charged; a fully covered block
that contributes to timeless_log_count from metadata does not consume row-
decode work.
The guard is available on all three direct SQLite/libSQL surfaces:
- hidden equality input
timeless_logs.max_work_entries; - optional sixth argument to
timeless_log_count; and - optional eighth argument to
timeless_log_values.
Earlier arities remain backward-compatible and unbounded. Embedded hosts that
need a hard policy should verify the corresponding query_surfaces flag from
timeless_capabilities() and always bind the cap. SQLite progress handlers and
sqlite3_interrupt() are observed between block reads/decodes, and a cancelled
connection remains reusable. LogsQL syntax, typed post-filters, result limits,
deadlines, and HTTP errors remain Rust signal-API responsibilities. The
SQL cookbook
contains executable direct-user recipes, including bounded rows/count/value
discovery and SQL-LOG-007 through SQL-LOG-009 for substring, exact and
presence predicates, and boolean composition. SQL-LOG-010 through
SQL-LOG-013 cover typed field discovery/projection, current-row filters,
empty and unique counts, lossless values, numeric aggregates, median, and
explicit-window rates using only the public logs table and SQLite JSON1.
Zero, negative, NULL, and non-integer equality guards fail, including a
supplied NULL positional TVF guard. With no equality guard the hidden column
projects NULL, so logs.max_work_entries IS NULL selects the compatible
unbounded form.
The Rust LogsQL layer now composes these public rows into case-sensitive
Unicode word and phrase filters, word/phrase prefixes, literal substring,
bounded RE2-compatible regexp, case-insensitive forms, full-message exactness,
typed numeric comparisons and open/closed ranges, logical retained-value
types, and NOT/AND/OR expressions with parentheses. Indexed service,
severity, time, and configured metadata constraints are pushed only when they
are safe top-level conjuncts; OR and NOT remain above the extension so
candidate pruning cannot change truth values. Regex and every decoded-row
predicate observe the request cancellation flag and max_work_entries before
a matching row may be returned.
The Rust LogsQL API implements VictoriaLogs-compatible pattern_match(...),
pattern_match_full(...), pattern_match_prefix(...), and
pattern_match_suffix(...). They search anywhere, require the complete field,
anchor at the beginning, or anchor at the end, respectively. Function names
are ASCII case-insensitive. A pattern is one quoted argument (or one simple
unquoted compound token):
pattern_match("request_id=<UUID>")
message:pattern_match_prefix("job <N>")
context.attempt:pattern_match_full("<N>")
* | filter peer:pattern_match_suffix("ip=<IP4>")
The seven recognized placeholders follow the pinned VictoriaLogs matcher, including its deliberately structural rather than validating interpretation:
| placeholder | matched shape |
|---|---|
<N> |
decimal digits, or an even-length hexadecimal token of at least four characters when hexadecimal letters are present |
<UUID> |
five <N> components separated by - |
<IP4> |
four <N> components separated by .; octet ranges are not validated |
<TIME> |
three <N> components separated by :, with optional . or , fraction |
<DATE> |
three <N> components separated by either - or / |
<DATETIME> |
<DATE>, T or space, <TIME>, and an optional Z or numeric offset |
<W> |
one Unicode General_Category Letter/Decimal_Number/underscore word or one valid quoted string; other number classes and combining marks are boundaries |
Unknown <...> text is literal. Empty any/prefix/suffix patterns match every
text value; an empty full pattern matches only textual empty. For this textual
operator only, missing and JSON null project as empty, strings use their exact
UTF-8 bytes, and retained booleans, numbers, arrays, and objects use compact
JSON text without changing the stored type. Typed equality, presence, and
numeric filters retain their stricter missing/null/type distinctions.
Pattern matching is bounded Rust API composition over the public logs rows.
It honors the existing work, result, response, deadline, and cancellation
limits and does not inspect private shadow tables. There is no claimed ordinary
SQL equivalent: SQLite LIKE and GLOB cannot faithfully reproduce the seven
token scanners, quoted-string escapes, Unicode word categories, and partial-
match restart behavior. This evidence does not justify a SQLite extension
primitive because it would not eliminate the already-required field decode.
Static exact membership uses in(v1, ..., vN). Values are case-sensitive,
quoted or unquoted, and matched against the same non-mutating rich textual
projection used by exact-prefix filters. in() matches nothing, a trailing
comma is accepted, and a quoted "*" is literal. Any standalone unquoted *
inside in, contains_any, or contains_all is instead a field-independent
no-op: it matches every bounded row even when the named field is absent.
The query-backed forms documented below use the same projection after their
bounded source query materializes exact values.
contains_all(v1, ..., vN) requires every non-empty static argument to match
the same field as a case-sensitive VictoriaLogs phrase. Letter, digit, and
underscore characters at either edge require Unicode word boundaries; quoted
phrases preserve their bytes. Arguments are independent, so
contains_all(ssh, "login fail") permits unrelated bytes between the two
matches. Duplicates do not change the result, a trailing comma is accepted,
and contains_all() or contains_all("") is a field-independent true
predicate. Missing and null project to empty text, strings retain their bytes,
and booleans, numbers, arrays, and objects use compact JSON text only while
matching—the retained metadata type is unchanged.
contains_any(v1, ..., vN) uses the same projection and phrase-boundary rules,
but succeeds when at least one static argument matches. contains_any()
matches nothing. Any empty-string argument is a field-independent true
predicate, so even missing:contains_any("") matches every bounded row.
Duplicates do not change results, a trailing comma is accepted, quoted stars
remain literal, function names are case-insensitive, and a non-empty list does
not match a missing field. For example, contains_any(error, "login failed")
matches either phrase; it does not require both.
equals_common_case(v1, ..., vN) and
contains_common_case(v1, ..., vN) reproduce VictoriaLogs' common-case
shortcut; they are not aliases for i(...). Each input phrase contributes its
whole-string Go-simple uppercase form plus every combination where each input
rune in Unicode category Lu independently remains unchanged or uses its
Go-simple lowercase mapping. For example,
equals_common_case(VictoriaMetrics) is equivalent to exact membership in
VictoriaMetrics, victoriaMetrics, Victoriametrics, victoriametrics,
and VICTORIAMETRICS. Other mixed-case spellings do not match. No Unicode
normalization is performed; titlecase Dž is not an uppercase toggle, while
the whole-uppercase candidate for Džx is DŽX.
The equals form applies full-value membership to the established rich
textual projection. The contains form applies the same generated candidates
with case-sensitive Unicode letter/digit/underscore phrase boundaries. Empty
lists are false. equals_common_case("") matches missing, null, and empty
textual projections; contains_common_case("") is a field-independent true
predicate. One trailing comma is accepted, function names are case-
insensitive, quoted "*" is literal, and an unquoted wildcard or malformed
separator fails explicitly.
VictoriaLogs rejects an input phrase with more than ten uppercase-letter
runes. Timeless preserves that limit and additionally bounds cumulative
request expansion to 8,192 distinct values and 4 MiB of parser state. Values
are sorted and deduplicated before existing exact/phrase predicates evaluate
public rows. Unqualified calls inspect _msg; arbitrary fields, field-prefix
groups, logical expressions, and current-row filter/where pipelines are
supported. The expansion changes neither retained rich values nor storage.
seq(v1, ..., vN) requires the non-empty phrases to appear in the selected
field in the declared order. Each phrase uses the same case-sensitive Unicode
letter/digit/underscore boundaries, and the next search begins only after the
preceding match ends. Unlike contains_all, order and duplicates therefore
matter: seq(open, close) differs from seq(close, open), and
seq(retry, retry) requires two non-overlapping occurrences. Empty arguments
are ignored; seq() and an all-empty list are field-independent true
predicates. One trailing comma is accepted, the function name is
case-insensitive, and unquoted wildcards or malformed separators fail
explicitly. The bare word seq remains an ordinary word filter.
Unqualified sequences inspect _msg; field:seq(...), field-prefix groups,
logical expressions, and filter/where pipelines inspect the selected
current-row textual projection. Strings retain their bytes, numbers and
booleans use their exact textual form, and retained arrays/objects use compact
JSON without changing stored types. VictoriaLogs flattens rich objects into
dotted string fields; Timeless deliberately retains native nested JSON and can
also sequence-match an explicitly selected parent object's compact projection.
field:in(query | fields value),
field:contains_any(query | fields value), and
field:contains_all(query | fields value) materialize a request-local value
list from a complete bounded LogsQL subquery. The subquery must end in exactly
one exact fields/keep field or one-field uniq; a final fields is
automatically deduplicated, and the ordinary implicit 100-row response limit
does not truncate the list. Nested selected fields use the canonical flattened
pipeline output name when their distinct values are materialized. Missing and null output project to empty text;
strings keep their bytes; numbers, booleans, arrays, and objects use the same
compact textual projection as static filters without changing retained data.
Empty query results make in and contains_any false and contains_all
true. Nested lists, logical expressions, current-row pipeline conditions, and
subquery pipeline order/limit are supported.
Every subquery and the outer query executes sequentially through the public
logs row/pipeline contract with one request timestamp. Equivalent subqueries
are evaluated once per request. The parser accepts at most eight nested levels
and 32 query-backed lists; materialized values, result cardinality, cumulative
decoded work, response/state bytes, and the complete request deadline remain
bounded. Internal execution uses the time remaining after parsing and source
resolution; a timeout envelope reports the configured request deadline rather
than that rounded remainder. The cache is released before the outer scan. No
LogsQL syntax, nested cursor, or private shadow table enters the extension.
field:json_array_contains_any(v1, ..., vN) selects only a retained JSON
array and succeeds when any top-level primitive element has the same exact
textual representation as a static candidate. Decoded strings compare by
case-sensitive bytes; numbers use their retained semantic JSON spelling;
booleans compare as true or false; and JSON null compares as null.
Nested arrays and objects are ignored rather than stringified. Missing fields,
scalars, objects, and empty arrays do not match. An empty candidate list is
false, while json_array_contains_any("") matches only an actual empty-string
array element. A trailing comma is accepted, duplicates are irrelevant,
function names are case-insensitive, a quoted "*" is literal, and an
unquoted * is invalid for this function. Query-backed candidates for this
JSON-array-specific function remain explicitly unsupported; shipped
LQL-F38 covers in, contains_any, and contains_all only.
Timeless intentionally applies this operation to its retained semantic JSON.
For example, a stored "a\u0062" array element is decoded to ab and matches
the candidate ab. The pinned VictoriaLogs implementation compares a raw
array lexeme in that shortcut and does not make that escaped-spelling match.
Timeless records this as a stronger typed-data interpretation instead of
retaining private lexical spellings or mutating storage.
Direct SQLite/libSQL users can express static membership with parameterized
IN and a field no-op by omitting the field predicate. Executable
SQL-LOG-015 and SQL-LOG-016 document both forms, including existing hidden-
column pruning for a declared string-only index key. SQL-LOG-017 uses public
json_each rows for exact top-level JSON-array primitive membership. These API
constructs do not require a private table or new extension primitive.
SQL-LOG-048 shows the bounded two-public-scan foundation for query-backed
exact membership, including cumulative work-budget guidance.
There is intentionally no contains_all, contains_any, seq, or complete
common-case SQL recipe.
Portable SQLite LIKE, GLOB, and instr cannot reproduce the required
Unicode-category word boundaries; seq additionally requires ordered,
non-overlapping phrase searches. Adding a storage primitive would not avoid
the public row decode already required for arbitrary rich fields. Direct SQL
users can compose intentionally looser instr() substring predicates when
that is their desired contract; those predicates are not labeled LogsQL
parity.
field:string_range(minimum, maximum) compares the complete textual field in
plain unsigned UTF-8 byte order. It includes minimum, excludes maximum,
and therefore makes equal or inverted bounds empty. The function accepts two
quoted or unquoted bounds, a trailing comma, case-insensitive function names,
message/service/arbitrary nested fields, and logical or pipeline composition.
Missing and null project to the empty string; strings keep their exact bytes;
retained numbers, booleans, arrays, and objects use compact JSON text only for
this predicate. Invalid arity, separators, wildcards, and unterminated input
fail instead of being ignored.
Executable SQL-LOG-019 implements the exact lower-inclusive/upper-exclusive
byte range for retained text plus missing/null-as-empty using only public
logs rows and SQLite JSON1. It casts both candidate and bounds to BLOB so
connection collation cannot alter byte order. Portable SQL intentionally
leaves non-string rich values to the Rust API. VictoriaLogs flattens nested
objects into dotted children before filtering; Timeless retains the object and
can compact-project a selected parent without losing its type. Both behaviors
are pinned, and no extension primitive is added because the operation already
uses the required public decoded rows.
field:len_range(minimum, maximum) measures the complete textual projection
in Unicode code points and includes both non-negative bounds. A multibyte
character such as é therefore has length one. Missing and null project to
length zero; strings retain their exact text; and retained numbers, booleans,
arrays, and objects use compact JSON text only while this predicate runs. An
inverted range matches nothing. Function names are case-insensitive, a
trailing comma is accepted, and message/service/arbitrary nested fields plus
logical and pipeline composition are supported.
Bounds follow the pinned VictoriaLogs unsigned grammar: quoted or unquoted
integers, base prefixes, digit separators, inf, byte-size expressions, and
duration expressions are accepted; negative values, unsuffixed fractions,
bad arity, missing separators, and unterminated calls fail explicitly.
Executable SQL-LOG-020 uses public logs rows, JSON1, and SQLite
length(TEXT) for exact retained-string and missing/null-as-empty semantics.
Portable SQL deliberately leaves rich-value projection and language grammar
to the Rust API. VictoriaLogs flattens objects before filtering, while
Timeless retains and can length-project the selected parent without losing
its type. No new extension primitive or storage format is involved.
left:eq_field(right), left:le_field(right), and
left:lt_field(right) compare two fields from the same bounded public log
row. An omitted left field selects the message. Field identifiers may be
quoted, function names are case-insensitive, and one trailing comma is
accepted. Message, level, service, and arbitrary dotted metadata are valid on
either side. _time is valid only on the right because a leading _time: is
reserved for the time-filter grammar. Logical expressions and ordered
filter/where pipelines compose these predicates; malformed arity,
separators, wildcards, and unterminated calls fail explicitly.
Both operands use the same non-mutating textual projection as other LogsQL
text filters. Missing and JSON null become empty text; strings retain their
bytes; numbers and booleans use compact JSON spelling; arrays and objects use
compact JSON only while the predicate runs; and a right-hand _time uses the
API's RFC3339 rendering in the table's configured timestamp unit. Equality is
exact textual equality, so 2 and 2.0 differ. Ordering first interprets
both projections as VictoriaLogs math values—decimal and base-zero numbers,
durations, byte sizes, RFC3339 timestamps, or IPv4 addresses—and otherwise
uses unsigned UTF-8 byte order. When both Timeless operands are retained JSON
numbers, exact JSON-number ordering takes precedence, preserving integers
beyond binary64 precision. A comparison with itself is therefore true for
eq_field and le_field, and false for lt_field, including when the field
is missing.
Executable SQL-LOG-021 uses only public logs rows and JSON1. It is the
complete retained-model SQL equivalent for eq_field and exposes the exact
bytewise fallback for le_field/lt_field. Portable SQL is not labeled as a
complete ordering equivalent because the language-specific math-value parser
remains in the Rust API. VictoriaLogs flattens objects before filtering;
Timeless retains and can compact-project a selected parent without losing its
type. Both operands already cross the same decoded public row, so a new
extension primitive would not avoid storage reads, decode, allocation, copy,
or row crossing.
A field selector ending in * applies its filter to every existing canonical
field whose name begins with the text before the wildcard. cmp_*:foo searches
cmp_left, cmp_right, and any other matching leaf until one succeeds;
*:foo and ""*:foo search every field. Prefixes may be quoted, including
"foo:bar:"*:exact(needle). _msg, _time, and level participate under
their canonical names. Retained metadata objects contribute dotted leaf names,
such as deployment.region, while arrays and null remain existing leaf values
and object parents are not implicitly flattened into matchable values.
Each atomic predicate expands independently. Consequently,
cmp_*:(bar AND foo) may satisfy bar in cmp_left and foo in cmp_right;
NOT negates the completed any-field result. A filter/where pipeline
enumerates the current projected row, so a preceding fields operation can
remove candidates. Expansion uses a single recursive path and stops at the
first match instead of allocating a row-wide field list. It observes request
cancellation at each retained node and remains bounded by the already-decoded
row and the API's storage-work, body, response, and deadline limits.
Wildcard field comparisons fail explicitly. VictoriaLogs currently treats a
left operand such as cmp_*:eq_field(right) as one literal nonexistent field
rather than expanding it, which can accidentally match missing/null/empty
projections. Timeless selects the strict behavior instead of preserving that
footgun. SQL-LOG-022 gives direct SQLite/libSQL users the executable public-
row field-set expansion for literal prefix selection and retained string/null
exactness. LogsQL parsing, word/phrase/range/rich-value semantics, RFC3339
_time projection, composition, limits, cancellation, and envelopes remain
in the Rust API. No extension primitive or storage-format change is involved.
_time:day_range[start, end] offset duration filters by a repeated UTC time-
of-day interval. Bounds accept HH:MM or HHMM; [/] include the exact
bound and (/) exclude it. The bounds are instants rather than minute
buckets, so a closed 12:00 includes exactly that timestamp and not the next
native tick. 24:00 clamps to the final nanosecond of the day, minute 60
normalizes into the following hour, and an inverted range is valid but empty.
The VictoriaLogs special case [00:00,00:00) selects the full day; other equal
half-open ranges are empty. Overnight wrapping is not implicit.
The optional offset is a signed VictoriaLogs compound duration and is added to
UTC before comparison. Timeless deliberately uses UTC when it is omitted. It
does not read the server process's local timezone, so the same request cannot
change with deployment location or daylight-saving state. Use an explicit
fixed offset when local wall time is intended. A pipeline filter reads the
current projected _time, so a preceding fields pipe can remove it.
SQL-LOG-023 gives direct SQLite/libSQL users the executable native timestamp
modulo and explicit-offset operation over bounded public rows, including open-
midnight normalization and millisecond/microsecond unit parameters. Clock and
duration grammar, logical/pipeline composition, errors, limits, cancellation,
and HTTP envelopes remain in the Rust API. The repeated daily predicate cannot
independently prune an arbitrary absolute time range, and ordinary SQL already
receives the timestamp, so no extension primitive or storage change is added.
_time:week_range[start, end] offset duration filters by a repeated UTC
weekday interval. Weekday names are case-insensitive and accept either short
(Sun through Sat) or full English spellings. Sunday is the beginning of
the linear range and Saturday is the end. Brackets are normalized before
comparison: an open start advances one weekday and an open end retreats one
weekday, both modulo seven. A resulting start above the end is valid and
empty; ranges do not otherwise wrap across the week boundary.
This preserves VictoriaLogs' edge cases. [Sun,Sun) and (Sat,Sun) normalize
to the full week, [Mon,Mon] selects Monday, and [Mon,Mon) is empty. The
optional signed compound offset is added to UTC before weekday selection.
Timeless uses deterministic UTC when it is omitted rather than inheriting the
server process's local timezone. Pipeline filters read the current projected
_time; removing that field earlier makes the predicate false.
SQL-LOG-024 gives direct SQLite/libSQL users the executable public-row
operation with millisecond/microsecond unit parameters, Euclidean pre-epoch
day handling, signed multi-day offsets, and explicit normalized weekday
bounds. LogsQL weekday/bracket/duration grammar, logical and pipeline
composition, errors, limits, cancellation, and envelopes remain in the Rust
API. The predicate cannot independently prune an arbitrary absolute timestamp
window, and ordinary SQL already receives the timestamp, so no extension
primitive or storage-format change is added.
LogsQL line comments begin with # outside double-quoted, single-quoted, and
raw-backtick literals and continue through the next LF. CRLF input is accepted;
the CR belongs to the comment and the LF remains the query boundary. A comment
marker may follow a token without intervening whitespace. Hashes inside quoted
field names and values are literal bytes.
Queries may span lines anywhere ordinary grammar permits, including between logical terms and pipeline stages. One optional terminal semicolon is accepted, including immediately before a trailing comment. A semicolon before remaining query text, multiple semicolons, a comment-only query, a dangling pipe, and a comment that removes a required pipeline argument fail explicitly. Lexical unterminated-quote and misplaced-semicolon errors include one-based source line and Unicode-character column positions.
The Rust API scans source once with memory bounded by the request-body limit.
Ordinary one-line queries remain borrowed without a normalization copy; a copy
is made only when comments or a terminal semicolon must be replaced. Replacement
preserves byte offsets and line boundaries, and the normalized LogsQL source
never enters SQLite. Direct SQLite/libSQL users write ordinary parameterized SQL,
so LQL-F40 has no separate SQL recipe or extension primitive.
delete, del, drop, and rm are case-insensitive aliases for the same
ordered row transform. They accept a comma-separated list of exact fields,
literal field prefixes ending in *, quoted field names, or the standalone
* that removes every field. The empty quoted field "" is VictoriaLogs'
message alias and therefore removes _msg. A missing exact field is a no-op,
repeated deletion is idempotent, and later pipeline stages observe only the
remaining fields. If no fields remain, the row is omitted rather than emitted
as {}.
Unquoted dotted names traverse Timeless's retained rich objects. Quoted names remain literal top-level keys even when they contain dots, commas, pipes, or asterisks. Prefixes compare case-sensitive canonical dotted paths and recurse through objects; arrays and scalars remain atomic and are removed only by their complete field path. Removing the last child prunes its now-empty parent. This preserves nested values without inventing VictoriaLogs' flattened storage model. Malformed commas, separated wildcards, leading wildcards, and embedded unquoted wildcards fail before storage work. Traversal is bounded by the decoded row/work limits and observes request cancellation.
SQL-LOG-025 gives direct SQLite/libSQL users an executable json_remove
projection for exact retained metadata paths. Direct SQL can omit ts,
message, or level from its projection, but LogsQL aliases, quoted/prefix
grammar, formatted _time, special-field deletion, recursive empty-parent
pruning, empty-row omission, composition, limits, cancellation, and envelopes
remain Rust API behavior. Public JSON1 already supplies the exact-path
foundation; no extension primitive or storage-format change is warranted.
Exact-build evidence over 8,192 retained rows measures exact plus nested- prefix deletion at 4.011/45.768 ms narrow/wide p95. That is 16.9%/17.6% above same-run word queries while response bytes are 22.4%/22.1% lower. Both paths read exactly one/four blocks, decode 1,024/8,192 entries, and read 235,778/1,914,055 payload bytes. The cost is bounded row mutation after the same public decode, not storage amplification.
The retained rich-log model intentionally differs from VictoriaLogs where
flattening would discard information. Numeric strings are not coerced, and
integer comparisons remain exact beyond 2^53. field:("") provides the
VictoriaLogs-compatible missing/null/empty predicate, while exact typed forms
continue to distinguish those states. field:* includes present zero, false,
arrays, and objects but excludes null and the empty string. value_type(...)
reports the stored logical JSON type rather than exposing private block
encoding choices. These decisions preserve embedded SQLite/libSQL value
fidelity without changing batching, compression, indexes, or on-disk formats.
Ordered LogsQL pipeline transforms run on the SQLite reader thread over the
same bounded public rows, so the HTTP deadline and cancellation flag cover
both storage and composition. fields/keep rebuild dotted nested paths and
a following filter/where evaluates the transformed row. field_names
discovers top-level response fields and counts presence, while
field_values returns deterministically ordered typed values and retains
missing as an omitted result field. Neither operation invents _stream or
_stream_id; those remain deferred until Timeless declares a stored stream
identity.
The API's typed statistic layer supports field/empty counts, exact and hashed
unique cardinality, typed unique values, lossless ordered values, numeric
sum/average/extrema/median, and interval rates. values(field) uses an object
with items and an exact missing count because an ordinary JSON array cannot
distinguish missing from a stored null. Numeric strings are not numbers. An
explicit positive limit bounds result or unique state according to the
operator; limit 0 means no operator-specific cap, but never disables
max_result_rows, max_work_rows, max_response_bytes, or the deadline.
Pipelines fail closed if the bounded public row set would be incomplete; they
do not aggregate a silently truncated prefix.
An unquoted selector {...} in a LogsQL filter or parenthesized subquery is
recognized as VictoriaLogs stream syntax and fails before storage with HTTP
422. The optional _stream: prefix has the same boundary:
{
"error": "unsupported_capability",
"reason": "unsupported_logsql",
"message": "LogsQL stream selector at line 1, column 1 is deferred: Timeless does not store a VictoriaLogs-compatible stream identity"
}Line and Unicode-character columns are one-based. Braces inside double-quoted,
single-quoted, or raw-backtick strings remain text, and braces after a #
comment marker are ignored with the rest of that line. Objects inside the
explicit case-insensitive rows({...}) inline-data source remain data for
join and union, including when such a source appears in a pipeline prefix
later discarded by generate_sequence. The check still applies to base
filters, current-row filter/where pipelines, and nested query text, so
unsupported syntax cannot silently broaden into a row scan.
VictoriaLogs derives a stream from the configured fields at ingestion,
canonicalizes the nonempty name/value pairs, hashes them into a tenant-scoped
identity, indexes it, and applies {...} before ordinary row predicates.
Timeless retains rich row metadata but not that declaration or identity.
service = :service against public SQL is therefore a useful ordinary filter,
not a stream-selector equivalent. Shipping LQL-F35 requires the complete
versioned stream contract named in the matrix and QSF-261; there is no honest
SQL recipe or extension primitive today.
VictoriaLogs also reserves _stream_id:<48-hex> for its tenant-scoped
internal stream identity. Timeless recognizes unquoted ASCII-case-insensitive
spellings, optional whitespace before :, and exact quoted _stream_id field
identifiers, then fails before storage with HTTP 422:
{
"error": "unsupported_capability",
"reason": "unsupported_logsql",
"message": "LogsQL _stream_id filter at line 1, column 1 is deferred: Timeless does not store a VictoriaLogs-compatible stream identity"
}This applies to exact IDs, static in(...) lists, query-backed in(...),
base filters, current-row filters, and nested query text. It does not reserve
an unqualified _stream_id word or a quoted message value, a nested metadata
path such as payload._stream_id, fields _stream_id, a comment, or an
_stream_id key inside explicit rows({...}) inline data. Those forms keep
their existing row/text behavior. The distinction prevents current retained
metadata from being mislabeled as VictoriaLogs' block identity.
There is no public SQL equivalent because no current table stores the tenant
prefix, canonical stream hash, or indexed identity. See LQL-F36,
QSF-263, and the versioned prerequisite shared with LQL-F35.
Direct SQLite/libSQL callers can inspect the actual work of one public log scan without subtracting cumulative process counters. Run both statements on the same connection and fully consume the first result:
SELECT ts, level, message, metadata
FROM logs
WHERE service = :service
AND ts >= :start_us AND ts <= :end_us
AND max_work_entries = :max_work_entries
ORDER BY ts;
SELECT query_total_ns, payload_bytes_read,
candidate_blocks, processed_blocks,
decoded_entries, processed_entries,
matched_entries, returned_entries,
values_read, timestamps_read
FROM timeless_log_query_stats('logs');The report is connection- and table-scoped and is consumed exactly once. A
new, failed, or cancelled scan clears an older report; a second read or fresh
connection fails explicitly. Six additional columns expose snapshot and
materialization timing, copied snapshot bytes, blocks skipped by an ordered
bound, buffered entries examined, and whether the snapshot used stable SQLite
locations. See executable SQL-LOG-026
for the complete schema and the fourteen-field LogsQL query_stats mapping.
Timeless codecs read one complete rich block payload rather than separately
addressable field-column files. Payload, block, entry, and logical-slot
counters therefore describe actual Timeless work; they do not pretend to be
VictoriaLogs per-column byte accounting. The Rust LogsQL API owns query
grammar, typed post-filter RowsFound, pipeline duration/composition, string
result values, limits, cancellation, and HTTP envelopes.
The Rust logs API implements the VictoriaLogs-compatible pipeline forms:
* | first
* | first 10 by (status desc, _time)
* | first 3 by (duration, _time) partition by (service, host)
rank as position
* | last 3 by (duration, _time) partition by (service, host)
rank as position
N defaults to one and must be positive. by is optional before the
parenthesized exact-field list; each field may specify asc or desc.
partition [by] (...) creates independent groups, and rank [as] field
inserts a one-based string rank that restarts in every partition (rank
defaults the field name to rank). Empty field lists and a trailing comma are
accepted where the pinned upstream grammar accepts them. Wildcard fields,
invalid counts, missing names, and trailing tokens fail before storage work.
last accepts exactly the same grammar and returns the reverse of first's
complete order. A per-field desc modifier reverses that field first, and the
operation-wide last direction reverses it again. Results are emitted from
last to first, and rank one names the first emitted row in each partition.
Missing and JSON null project to empty text. Sort coercion follows pinned
VictoriaLogs order: exact signed integer, exact unsigned integer, RFC3339
timestamp, numeric/duration/byte value, then natural UTF-8 byte order. Sort
directions apply per field. Partition keys use length-framed textual values
and partitions have deterministic encoded-key order. Equal sort keys use the
original public-row order as Timeless's stable tie-break because upstream does
not promise an equal-key order. With no by fields, first observes the
current pipeline schema: a preceding fields or delete changes the encoded
row used for comparison. Timeless preserves numbers, booleans, arrays,
objects, nulls, and nested metadata in its response rather than flattening
them to strings.
The operation consumes only bounded public timeless_logs rows. Input and
output are capped by max_work_rows and max_result_rows; sort keys,
partitions, indexes, and selected rows are charged to the existing
max_response_bytes memory budget. Cancellation is observed while keys are
built, comparisons run, and output is assembled. No private table or new
extension primitive is used.
Direct SQLite/libSQL users can implement bounded per-partition numeric
selection with row_number() over public rows. Executable
SQL-LOG-027
includes the parameterized statement, timestamp units, order, missing/null,
rank-type, and result-bound contract. It deliberately does not claim full
LogsQL natural collation or exact cross-type integer coercion from ordinary
SQLite REAL.
Executable
SQL-LOG-028
provides the corresponding descending window-rank statement and documents the
same boundary.
The Rust logs API implements VictoriaLogs-compatible random row sampling at the current pipeline position:
* | sample 4
service:="api" | fields service, context | sample 100
* | sample 1 | stats count() as total
sample N retains approximately one of every N input rows. N uses the
pinned VictoriaLogs positive-unsigned grammar: decimal and base-zero integers,
quoted values, byte-size and duration suffixes, and inf/+inf are accepted;
zero, negative, invalid-octal, unsuffixed fractional, missing, extra, and
parenthesized values fail before storage work. Commands are case-insensitive.
sample 1 is an exact no-op.
Every request owns a fresh random generator. Like VictoriaLogs, the evaluator
draws exponentially distributed gaps whose mean yields a 1/N selection rate;
it does not choose deterministic every-Nth rows. Retained rows remain in input
order and preserve their complete current rich JSON values. When sampling is
the first pipeline stage, discarded public rows are removed before metadata
JSON materialization. At later positions it samples the already transformed
current rows, preserving ordered pipeline composition.
The required public scan remains bounded by max_work_rows. Sampled output is
bounded by max_result_rows and max_response_bytes, and cancellation is
checked while rows are compacted in place. Sampling changes no stored row,
block, batch, index, compression, or durability contract.
Executable
SQL-LOG-049
provides a parameterized public-row SQLite/libSQL equivalent using independent
Bernoulli draws. It implements the public 1/N random-subset contract without
claiming VictoriaLogs' private exponential-gap RNG sequence. Ordinary SQL is
sufficient, so no extension primitive or private storage access is used.
Exact release-build evidence over 8,192 rich rows compares sample 4 with the
exact sample 1 control before the same scalar count. Narrow p50/p95/p99 is
3.027/3.657/3.910 ms versus 3.140/3.307/3.447 ms; wide is
25.179/26.060/26.533 ms versus 32.412/33.206/33.678 ms. Wide sample p95 is
21.5% lower because only retained rows reach metadata JSON materialization.
Narrow p95 is 10.6% higher even though request-attributed API time is 4.4%
lower, so the small-query tail is retained as endpoint variation. Every pair
performs identical bounded public work; the evidence harness rejects native
count or any control with different requested entries, blocks, decoded rows,
payload bytes, matches, or returned rows.
The Rust logs API implements frequency ranking over the current pipeline row:
* | top by (service)
* | top 5 service, level hits as total rank as position
* | filter level:=error | top 10 by (service) rank
The default limit is ten. by is optional; fields may be a parenthesized or
bare comma-separated exact list. hits [as] field renames the required string
hit count, while rank [as] field adds a one-based string rank. Default or
explicit result names gain trailing s characters until they no longer
collide with a selected field. Commands and modifiers are case-insensitive;
zero/fractional limits, empty/wildcard fields, missing names, unseparated
fields, and trailing tokens fail before storage work.
Every selected value uses the LogsQL textual projection. Missing, JSON null, and empty strings therefore share one empty group; its selected field is omitted from response JSON while hits/rank remain. Strings are unquoted and numbers, booleans, arrays, and objects use their retained textual forms. A multi-field vector is framed structurally, so different tuples cannot collide. Groups order by hits descending and then projected key ascending. The output is a summary and does not mutate or flatten stored rich metadata.
Input rows and unique groups are bounded by max_work_rows; retained keys and
group/sort state are charged to max_response_bytes; the requested output is
bounded by max_result_rows; and cancellation is checked during grouping,
sorting, and assembly. The operation reads only the public log rowset.
Executable
SQL-LOG-029
provides the single-field public GROUP BY, deterministic ordering, and
window-rank equivalent. Multi-field query grammar, current-row composition,
name collision policy, limits, cancellation, and HTTP envelopes remain API
behavior. No new extension primitive or private storage access is required.
uniq emits one textual row for each structural key selected from exact
current-row fields. facets instead discovers every current-row field and
emits its most frequent nonempty textual values:
* | uniq service, level with hits limit 20
* | fields service, level, context | facets 5
* | facets max_values_per_field 1000 max_value_len 128 keep_const_fields
For uniq, optional filter is a case-sensitive single-field substring,
hits is optional, zero means no language-specific limit, and positive-limit
overflow resets retained hits to string "0". Missing, null, and empty share
one empty key component; the empty response field is omitted. Timeless selects
structural keys bytewise so tuples cannot collide and results are repeatable
even though VictoriaLogs does not promise its hash-map subset or order.
For facets, the defaults are ten results per field, at most 1,000 unique
textual values tracked per field, and at most 128 UTF-8 bytes per value. Empty
values are ignored. A field with any longer value or excessive cardinality is
omitted entirely. A single value appearing in every selected row is omitted
unless keep_const_fields is present. Objects become dotted leaves and arrays
remain atomic JSON text. Results are deterministic by field name, hits
descending, and bytewise value. Modifiers are case-insensitive, reorderable,
and repeatable; matching VictoriaLogs v1.52.0, positive fractions are
truncated before use.
Both operations preserve rich stored rows and enforce hard input/state/result/
response limits plus cancellation. Executable
SQL-LOG-030
provides direct SQLite/libSQL grouping for uniq.
SQL-LOG-031
provides recursive JSON1 field discovery, canonical _time/_msg/level
projection, cardinality/length/constant exclusion, and per-field window ranks
for facets. Neither operation requires a new extension primitive or private
storage access.
coalesce writes the first nonempty textual source value to an exact
destination field:
* | coalesce(trace_id, request_id) default unknown as correlation_id
* | coalesce(context.*, service) as context.primary
* | fields error, message | coalesce(error, message)
Sources are parenthesized and may be exact fields, *, or suffix-star prefix
filters. Source filters are evaluated left to right and expanded names are
de-duplicated. Missing, JSON null, empty strings, and exact rich-object parents
are skipped; object leaves participate through dotted names. Strings remain
unquoted, numbers and booleans become text, and arrays remain one compact JSON
text value. Wildcard expansion is deterministic by bytewise flattened field
name. A trailing source comma is accepted to match VictoriaLogs.
The destination defaults to _msg. default value supplies a textual value
when no source is nonempty, and as field chooses an exact destination.
Timeless retains an explicitly empty destination in the rich JSON result;
VictoriaLogs omits empty-valued columns when serializing streams. If a nested
destination would replace a retained scalar parent, Timeless returns HTTP 422
with reason field_conflict and leaves the row unchanged.
Work, temporary path/de-duplication state, results, response bytes, and
cancellation are bounded. Executable
SQL-LOG-032
shows the ordinary public CASE/NULLIF/COALESCE equivalent for exact
metadata paths. No extension primitive or private table is needed.
copy (alias cp) preserves source fields while cloning them to one or more
destinations:
* | copy trace_id as correlation_id
* | cp context.* as copied.*, copied.attempt as retry_attempt
* | copy service saved, host service, saved host
as is optional. Comma-separated pairs execute left to right, so later pairs
observe earlier copies and can form chains or swaps. Sources and destinations
may be exact fields, *, or suffix-star prefix filters. A wildcard source is
snapshotted at the start of its pair and expands recursively flattened leaves
in bytewise field-name order; arrays remain atomic values. Prefix destinations
replace the matched source prefix. Copying many wildcard sources to one exact
destination is deterministic last-write-wins. A missing wildcard source is a
no-op.
Exact copies preserve JSON strings, numbers, booleans, arrays, null, and empty
strings without deleting or coercing the source. A missing exact source or an
exact rich-object parent produces an explicit empty string, matching the
upstream flattened-column view; copy an exact dotted leaf or use a prefix to
clone object contents. An exact source paired with a wildcard destination
uses the literal destination name, including *, matching VictoriaLogs.
When prefix removal yields an empty destination suffix, it creates a literal
empty field distinct from _msg; exact quoted "" still names _msg.
Existing compatible scalar destinations are overwritten. A destination that
would replace a retained object or descend through a scalar fails with HTTP
422 reason field_conflict, preserving rich-row fidelity. Source traversal,
temporary cloned values and paths, result rows, response bytes, and
cancellation are bounded. Executable
SQL-LOG-033
shows the public JSON1 equivalent for one exact retained metadata source and
one exact top-level destination. Sequential/wildcard language composition
remains in the Rust API; no extension primitive or private table is used.
rename (alias mv) moves fields within the current response row:
* | rename trace_id as correlation_id
* | mv context.* as moved.*, moved.attempt as retry_attempt
* | rename service saved, host service, saved host
as is optional. Comma-separated pairs execute left to right, so later pairs
observe earlier removals and destinations and can implement chains or swaps.
Sources and destinations may be exact fields, *, or suffix-star prefix
filters. Each wildcard source snapshots the current recursively flattened
leaves in bytewise field-name order. Arrays remain atomic. All sources for one
pair are removed before its destinations are inserted. Prefix destinations
replace the matched source prefix; multiple wildcard sources moved to one
exact destination are deterministic last-write-wins. An unmatched wildcard
source is a no-op.
Exact strings, numbers, booleans, arrays, null, and empty strings retain their
JSON types. Present leaves are removed from the response and empty rich
parents are pruned, but the stored row remains immutable. A missing exact
source or exact rich-object parent produces an explicit empty destination;
the object remains because VictoriaLogs' flattened view has no parent column.
Rich empty objects likewise have no wildcard leaf and are retained. An exact
source paired with a wildcard destination uses the literal destination name,
including *. When prefix removal yields an empty suffix, the destination is
a literal empty field distinct from _msg; exact quoted "" still names the
message.
Compatible scalar destinations are overwritten. A destination that would
replace a retained object or descend through a scalar fails with HTTP 422
reason field_conflict. Traversal, temporary moved values and paths, result
rows, response bytes, and cancellation are bounded. Executable
SQL-LOG-034
shows the public JSON1 foundation for one exact top-level move. The Rust API
owns nested-parent pruning, wildcard and sequential composition, strict
errors, and hard limits; no extension primitive or private table is used.
format interpolates current-row fields into a textual result:
* | format "request from <client_ip>: <_msg>"
* | format if (level:=error) '<uc:service> <q:_msg>' as summary
* | format '<duration_seconds:elapsed>' as elapsed_seconds keep_original_fields
* | format '<urlencode:user>' as encoded_user skip_empty_results
The pattern may be quoted or a single unquoted token. Literal prefixes decode
HTML entities. <field> uses recursively retained rich paths and textual
projection: strings remain unquoted, numbers and booleans use JSON spelling,
arrays use compact JSON, and missing/null values are empty. <_>, <*>, and
<> are explicit empty placeholders; wildcard field references are rejected.
An unknown option is a plain interpolation for VictoriaLogs compatibility.
The supported options are uc, lc, q, urlencode, urldecode,
hexencode, hexdecode, base64encode, base64decode, hexnumencode,
hexnumdecode, time, duration, duration_seconds, and ipv4. Invalid
codec inputs retain their source text, except hexdecode preserves invalid
byte pairs while decoding valid pairs exactly as the pinned processor does.
uc/lc use simple one-codepoint Unicode mappings. time accepts the exact
VictoriaLogs signed integer, fractional, and scientific Unix s/ms/us/ns
heuristic and emits trimmed nanosecond RFC3339 UTC. duration accepts signed
nanoseconds; duration_seconds accepts the established human-duration
grammar.
The destination is _msg unless as exact_field is present. if (...)
formats only matching rows; if () matches every row. A nonempty existing
destination is retained by keep_original_fields, and by
skip_empty_results only when the new result is empty. Timeless preserves an
explicit empty destination in rich JSON, whereas VictoriaLogs stream JSON
omits empty-valued columns. Existing scalar destinations are overwritten. A
destination that would replace a retained object or descend through a scalar
fails with HTTP 422 reason field_conflict and leaves durable storage
unchanged.
Pattern/source traversal, transform expansion, temporary output, result rows,
response bytes, and cancellation use the hard request limits. Executable
SQL-LOG-035
shows a public JSON1/printf equivalent for two exact metadata paths. The
Rust API owns LogsQL syntax, arbitrary placeholders and codecs, conditions,
destination preservation, errors, and limits; no extension primitive or
private table is used. Exact-build evidence measures 3.297/39.353 ms
narrow/wide p95 versus 3.090/35.941 ms for byte-identical same-scan controls;
QSF-171 accepts the +6.7%/+9.5% bounded formatting cost.
math and its alias eval calculate one or more binary64 expressions and
write string results into the current response row:
* | math duration + 10e9 as adjusted_ns
* | eval attempts + 1 next_attempt, next_attempt * backoff as delay
* | math round(bytes / 1KiB, 0.01) as kib
* | math invalid default 0 as safe_value
Comma-separated entries execute left to right, and a later entry can read an
earlier destination. as is optional. If the destination is omitted, the
canonical expression—including necessary parentheses—becomes the field name.
Only exact destinations are accepted.
From tightest to loosest, the binary operators are ^, *///%,
+/-, &, xor, or, and default. All associate left, including
power. Unary plus/minus and explicit parentheses are supported. Available
functions are abs, ceil, exp, floor, ln, max, min, now,
rand, and round. max/min require at least two arguments and skip NaN
operands in evaluation order. round(value) rounds to an integer;
round(value, nearest) uses the pinned VictoriaLogs decimal-scale rule.
now() returns a pipeline-invocation Unix timestamp in nanoseconds and
rand() returns a value in [0,1).
Numbers may be decimal, base-zero, scaled, durations, byte sizes, RFC3339
timestamps, or IPv4 addresses; durations and timestamps become nanoseconds.
Current-row fields use the same coercion. Missing, null, empty, arrays,
objects, and invalid text become NaN. default replaces only NaN—not
infinity. Results use fixed, non-exponent strings, including NaN, +Inf,
and -Inf. Bitwise operands follow the pinned VictoriaLogs unsigned
conversion even for negative, nonfinite, or out-of-range values, so results
do not depend on Rust target cast behavior.
Rich source values are never changed. Scalar destinations are overwritten;
replacing an object or descending through a scalar returns HTTP 422 reason
field_conflict. Parser AST size/nesting, evaluated work, temporary state,
result rows, response bytes, and cancellation use the hard request limits.
Executable SQL-LOG-036
shows a parameterized public JSON1 equivalent for ordinary arithmetic over
two exact numeric metadata fields. SQL deliberately returns NULL for
invalid inputs instead of SQLite's misleading CAST('bad' AS REAL) = 0;
the complete LogsQL grammar, coercion chain, functions, sequential mutation,
and error envelopes remain Rust API behavior. No extension primitive or
private table is used.
Exact-build math evidence measures 3.357/39.127 ms narrow/wide p95 while
returning 64 rows, versus 3.292/37.655 ms for byte-identical same-scan
controls. The +2.0%/+3.9% p95 and +2.4%/+9.6% internal API cost follows the
same one/four candidate blocks, 1,024/8,192 decoded entries, and
235,778/1,914,055 payload bytes. QSF-173 accepts this bounded expression
cost above the unchanged public storage boundary.
len measures the byte length of one current-row field and writes the decimal
result as a string:
* | len(_msg) as message_bytes
* | len unicode byte_length
* | len(nested.value)
* | len(host)
The command and as are case-insensitive. Parentheses and as are optional;
the destination defaults to _msg, including the accepted len(field) as
form. An empty quoted source or destination is the canonical _msg alias.
Only exact quoted or dotted source and destination fields are accepted, and
sequential pipes observe earlier destinations.
Length is measured in UTF-8 bytes, so len("ßİ") is four even though the
text contains two Unicode codepoints. Strings use their decoded bytes;
booleans and numbers use their textual representation; arrays use compact
JSON. Missing fields, explicit null, empty strings, and exact retained object
parents have length zero, matching VictoriaLogs' flattened query view. A
nested object leaf remains addressable. Canonical _msg, _time, and
level fields are measured from their current rendered values. Rich source
values and durable storage remain unchanged.
An exact scalar destination is overwritten. A destination that would replace
a retained object or descend through a scalar fails with HTTP 422 reason
field_conflict. Array traversal work, temporary result/path state, result
rows, response bytes, and cancellation use the hard request limits.
Executable
SQL-LOG-037
uses only public logs rows plus SQLite JSON1 and
length(CAST(value AS BLOB)); the BLOB cast is required because SQLite
length(TEXT) counts codepoints. Grammar, canonical/current-row fields,
sequential destinations, limits, cancellation, and envelopes remain Rust API
work. No extension primitive, private table, or storage-format change is
needed.
Exact-build len evidence measures 3.785/40.724 ms narrow/wide p95 while
returning 64 rows, versus 3.620/36.622 ms for byte-identical same-scan
controls. The +4.6%/+11.2% p95 and -1.0%/+12.6% internal API variation
follows the same one/four candidate blocks, 1,024/8,192 decoded entries, and
235,778/1,914,055 payload bytes. QSF-175 accepts the bounded row-local byte-
length work above the unchanged public storage boundary.
hash computes the VictoriaLogs-compatible integer hash of one current-row
field and writes the decimal result as a string:
* | hash(user_id) as user_hash
* | hash nested.value value_hash
* | hash(_msg)
The command and as are case-insensitive. Parentheses and as are optional;
the destination defaults to _msg, and empty quoted source or destination
names alias _msg. Only exact quoted or dotted fields are accepted. Later
pipeline stages see earlier hash destinations.
The algorithm is seed-zero xxHash64 with the result masked by (1 << 53) - 1,
matching VictoriaLogs' exactly representable binary64 integer domain. Strings
use their decoded bytes; booleans and numbers use their textual spelling;
arrays use compact JSON. Missing fields, explicit null, empty strings, and
exact retained object parents hash the empty byte string. Nested object leaves
and canonical current _msg, _time, and level values remain addressable.
Native rich sources and durable storage are not changed.
Compact arrays are traversed under the request work/nesting allowance and
streamed directly into xxHash64. Temporary state, result rows, response bytes,
and cancellation use the shared hard limits. Replacing a retained object or
descending through a scalar fails with HTTP 422 reason field_conflict.
There is deliberately no claimed portable SQL equivalent: core SQLite/libSQL
does not provide xxHash64, and signed-integer SQL expressions cannot reproduce
its unsigned wrapping operations honestly. Applications may register an
xxHash64 UDF for their own bounded public logs queries, but that is not a
timeless-libsql contract. The Rust logs API owns this row-local transform;
no extension primitive, private table, or storage-format change is involved.
Exact-build evidence measures 3.455/36.785 ms narrow/wide p95 versus
3.481/36.223 ms for byte-identical public-storage-work controls. Both pairs
read one/four blocks, decode 1,024/8,192 entries, and read
235,778/1,914,055 payload bytes per query. Decimal hashes make the 64-row
response 502/506 bytes larger. QSF-210 accepts the -0.7%/+1.6% p95
variation and keeps hashing above the unchanged public storage boundary.
collapse_nums normalizes eligible decimal and hexadecimal tokens in one
current-row field to <N>:
* | collapse_nums
* | collapse_nums at message_template
* | collapse_nums if (service:=api) at message_template prettify
The default target is _msg. if (...) is optional and evaluated against the
current row; at accepts one exact quoted or dotted field; and prettify must
be the terminal modifier. Keywords are case-insensitive. Invalid ordering,
wildcards, attached suffixes, missing arguments, and trailing syntax fail
explicitly.
The scanner follows VictoriaLogs' byte-exact boundaries. Decimal runs may have
any length. Hexadecimal candidates containing a–f must be at least four
bytes and even-length, avoiding common short words. ASCII letters, digits, and
underscore delimit tokens except for the pinned underscore, version, time,
duration, and unit boundary characters; non-ASCII bytes delimit candidates.
prettify then recognizes collapsed UUID, IPv4, time, date, and datetime
shapes, including fractional seconds and numeric or Z timezones.
Strings are transformed directly; numbers and booleans use their textual spelling; arrays use compact JSON; and missing, null, and exact object parents project as empty text. Timeless preserves a native typed value when the projected text does not change, while an actual transformation writes a string to the same current-row field. Sequential pipes see the updated string. Stored rows remain immutable, and work, temporary bytes, output, nesting, deadline, and cancellation remain under the shared query limits.
There is deliberately no claimed portable SQL equivalent. Core SQLite/libSQL
has no tokenizer or replacement scalar with the exact boundary, hexadecimal,
and ordered prettification behavior. A recursive character CTE or ordinary
replace() chain would be a different language. The Rust logs API owns this
bounded row-local transform; no private table, extension primitive, or storage
format is involved.
Exact-build evidence measures 3.135/34.525 ms narrow/wide p95 versus
3.143/36.735 ms for identical-output, same-public-work controls. Both pairs
return 64 rows and 1,536 bytes, read one/four blocks, decode 1,024/8,192
entries, and transfer 235,778/1,914,055 extension payload bytes per query.
QSF-212 treats the -0.3%/-6.0% endpoint-tail difference as bounded run/API
variation after the unchanged public scan and keeps number collapsing in the
Rust language layer.
decolorize removes the exact VictoriaLogs ANSI Control Sequence Introducer
form from _msg or one exact current-row field:
* | decolorize
* | decolorize "rendered message"
* | decolorize nested.output | format '<nested.output>' as rendered
The command is case-insensitive. An omitted or empty quoted field means
_msg; quoted and dotted exact fields are valid. Wildcards, prefix selectors,
parentheses, comma-separated fields, attached suffixes, and extra tokens are
malformed rather than silently ignored.
The scanner is byte-exact. It removes ESC [; zero or more parameter bytes
in 0x30..0x3f; zero or more intermediate bytes in 0x20..0x2f; and one
optional final byte in 0x30..0x7e. An incomplete CSI is removed. If the next
byte is outside those classes, it remains in the output. Other escape families
such as OSC and DCS are not CSI and remain unchanged. This is intentionally
the pinned VictoriaLogs behavior, not a claim to strip every terminal escape
language.
Strings use decoded bytes; numbers and booleans use their textual spelling; arrays use compact JSON; and missing, null, and exact object parents project as empty text. Timeless preserves native missing/null/number/boolean/array/ object states when no CSI is found. A real removal writes a string to the same request-owned current-row field, and subsequent pipes observe it. Stored rich rows remain immutable. Cumulative work, temporary bytes, result/response size, deadline, cancellation, destination conflicts, optimize, shutdown, and reopen use the shared hard contracts.
SQL-LOG-050
is an executable direct-SQL foundation. It scans bounded public logs rows
and uses BLOB positions plus an optimized recursive state machine so UTF-8,
embedded NUL, byte classes, incomplete sequences, and invalid-final behavior
remain exact. The statement returns the flattened textual projection. The
Rust API adds LogsQL grammar, native no-op preservation, current-row
composition, limits, cancellation, and response envelopes. Because every row
already crossed the public storage boundary, adding a language-specific
extension primitive would not avoid a block read, decode, payload transfer,
or row crossing.
Exact-build evidence measures 3.101/36.385 ms narrow/wide p95 versus
3.169/34.872 ms for identical-output format controls. Both pairs return 64
rows and 1,536 bytes, read one/four blocks, decode 1,024/8,192 entries, and
transfer 235,778/1,914,055 extension payload bytes per query. The candidate
also constructs the colored current-row value being stripped. QSF-214
accepts the -2.2%/+4.3% endpoint-tail and +1.4%/+3.7% request-attributed mean
differences as bounded language-layer work after the unchanged public scan.
split divides one current-row field by a literal separator and writes a
compact JSON-array string:
* | split ","
* | split "::" from source as parts
* | split "," source parts
* | split "" from unicode as runes
The command and optional from/as keywords are case-insensitive. Source
defaults to _msg; destination defaults to the source, so omitting as
overwrites only the request-owned current row. Both keywords may be omitted
in the VictoriaLogs shorthand. Exact quoted and dotted fields are valid.
Wildcards, prefixes, comma-separated operands, parenthesized call syntax,
attached suffixes, missing operands, and trailing tokens are malformed. Quote
a separator named from or as.
Splitting is literal and non-overlapping. Leading, trailing, and consecutive
separators retain empty elements. If a nonempty separator is absent, the
whole source is one element. An empty separator emits one element per Unicode
scalar value. Consequently, an empty source becomes [""] with a nonempty
separator and [] with an empty separator. Output is a string containing
compact JSON, not a native retained array; downstream json_array_len parses
it normally. VictoriaLogs wire spelling is preserved exactly, including
\u003c for < and \u0027 for apostrophe.
Strings split directly. Numbers and booleans use their textual spelling;
arrays use compact JSON; and missing, null, and exact object parents project
as empty text. When source and destination differ, the original native value
remains present. Nested destination paths preserve siblings and reject an
unsafe object/scalar replacement with HTTP 422 field_conflict. Durable
public rows are never changed. Work per row and emitted piece, temporary
bytes, result/response size, deadline, cancellation, optimize, shutdown, and
reopen use the shared hard contracts.
SQL-LOG-051
is the executable direct-SQL foundation. It uses only bounded public logs
rows, a recursive CTE, and JSON1, and pins literal, empty-piece, and
Unicode-scalar behavior. SQLite JSON1 may spell < and apostrophe literally
while preserving the same decoded array; the Rust API supplies the exact
VictoriaLogs wire text plus LogsQL grammar, rich current-row mutation,
limits, cancellation, and envelopes. Since every source row already crossed
the public storage surface, an extension split primitive would not avoid a
block read, decode, allocation, payload transfer, or row crossing.
QSF-215 records the exact pinned semantic and ownership boundary.
QSF-216 measures 3.219/3.481/4.063 ms narrow and
37.529/38.655/40.113 ms wide p50/p95/p99, versus
3.078/4.786/4.878 and 37.964/40.047/40.151 ms for identical-output controls.
Every pair has the same 64 rows, 1,984 response bytes, candidate blocks,
decoded entries, extension payload bytes, and public rows. Split p95 is
27.3%/3.5% lower; request-attributed API means are 3.1% higher/1.4% lower.
These are accepted bounded run/API differences after the unchanged public
scan, not evidence for a split-specific extension primitive.
drop_empty_fields removes empty fields from each current pipeline row:
* | drop_empty_fields
* | fields case, optional, nested | drop_empty_fields
* | format "" as transient | drop_empty_fields
The command is case-insensitive and accepts no arguments. JSON null and an
empty string are empty. Missing fields are already absent. Zero, false,
nonempty strings, and arrays—including []—are retained without changing
their types. Rich objects are traversed recursively: empty leaves and newly
empty parents are removed, but arrays are atomic and their elements are not
fields. If a prior fields, delete, format, or other transformation leaves
no fields at all, the row is omitted. Later pipeline stages observe the
pruned row. Durable stored metadata is never changed.
Traversal is in place over request-owned public rows. JSON nesting is capped at 128 levels; every visited row/value consumes the hard work allowance; cancellation is checked before and during traversal; and final result rows and response bytes use the shared request limits. Invalid arguments, parentheses, aliases, and trailing tokens fail as malformed LogsQL.
Executable
SQL-LOG-038
uses only public logs rows and SQLite JSON1 to remove one known null/empty
metadata path. A fixed-schema embedded application can repeat that expression
for its known fields. Dynamic field discovery, canonical _msg/_time/level
handling, recursive empty-parent and all-empty-row pruning, resource limits,
cancellation, and HTTP envelopes remain Rust API behavior. No extension
primitive or private storage table is used.
Exact-build drop_empty_fields evidence measures 4.542/38.151 ms narrow/wide
p95 while returning 64 rows, versus 6.994/35.779 ms for byte-identical same-
scan controls. The -35.1%/+6.6% p95 and -3.3%/+11.0% internal API variation
follows the same one/four candidate blocks, 1,024/8,192 decoded entries, and
235,778/1,914,055 payload bytes. Responses are byte-identical. QSF-177
accepts the bounded in-place rich-row traversal above the unchanged public
storage boundary.
replace substitutes literal, non-overlapping substrings in one exact
current-row field:
* | replace ("_", "-")
* | replace ("_", "-") at host limit 1
* | replace if (kind:=admin) ("secret", "***") at password
* | replace if () ("a,b", "c|d") at "quoted field" limit 0
The command, if, at, and limit keywords are case-insensitive. Two
parenthesized literal substrings are required; quoted values may contain
spaces, commas, pipes, and Unicode. This is byte-for-byte substring matching,
not regular-expression replacement. The target defaults to _msg and may be
one quoted or dotted exact field. limit 0 or an omitted limit replaces every
non-overlapping occurrence; a positive limit replaces only the first N.
An empty old substring is a no-op. Optional if (...) evaluates against the
current row before replacement, and later pipeline stages observe the result.
VictoriaLogs stores a flattened textual view. Timeless projects strings,
lowercase booleans, numbers, and compact JSON arrays for replacement while
treating a missing field, null, or exact object parent as empty. When the old
literal does not match—or is empty—the original rich native value remains
unchanged. An actual replacement produces a string in the query result only;
durable metadata and canonical fields are never mutated. This retained-model
rule preserves more information without changing the observable replacement
text. Sequential replace pipes see prior transformations.
Parsing is strict. In particular, Timeless rejects attached
replace(foo,bar) syntax. The pinned VictoriaLogs replace-pipe parser rejects
that spelling too, although its whole-query endpoint can ambiguously accept it
as an unrelated filter; Timeless does not silently reinterpret malformed pipe
syntax. Wildcard targets, invalid or leading-zero limits, wrong arity, and
trailing tokens also fail as malformed LogsQL.
Projected arrays, matches, generated text, field paths, work, result rows,
response bytes, and cancellation use the hard request limits. Executable
SQL-LOG-039
uses only public logs rows, SQLite JSON1, and core replace() for an
all-occurrence exact-field equivalent. Conditional, first-N, current-row,
rich-preservation, limit, cancellation, and HTTP-envelope behavior remains in
the Rust API. No extension primitive, private storage table, or durable format
change is needed.
Exact-build literal-replacement evidence measures 3.240/3.520/3.576 ms
narrow and 36.716/37.711/37.948 ms wide p50/p95/p99 while returning 64 rows
and 1,600 response bytes. Byte-identical same-scan controls measure
3.261/3.908/4.406 and 35.734/36.882/37.846 ms. The -9.9%/+2.2% p95 and
-3.6%/+2.7% internal API variation follows the same one/four candidate
blocks, 1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and
128/8,192 public rows. QSF-179 accepts this bounded literal row transform
above the unchanged public storage boundary.
replace_regexp substitutes non-overlapping RE2-family matches in one exact
current-row field:
* | replace_regexp ("[/ ]", "-")
* | replace_regexp ("(?P<name>[a-z]+)-(?P<id>[0-9]+)", "${id}:${name}") at host
* | replace_regexp if (kind:=admin) ("secret=([^ ]+)", "secret=***") limit 1
* | replace_regexp ("^|$", X) at host limit 0
The command, if, at, and limit keywords are case-insensitive. Exactly
two parenthesized arguments are required. The target defaults to _msg and
may be one quoted or dotted exact field. A missing or zero limit replaces all
non-overlapping matches; a positive limit replaces only the first N.
Optional if (...) observes the current row before replacement, and later
pipeline stages observe the transformed value.
Patterns use the pinned VictoriaLogs/Go RE2-family contract: matching is
case-sensitive unless an inline flag changes it, dot matches a newline by
default, (?-s) restores single-line dot behavior, and backreferences and
lookaround are rejected. An empty pattern matches UTF-8 boundaries, including
the start and end of a nonempty value. An empty source remains a no-op, as it
does upstream. Patterns are compiled once per request with a one-MiB compiled
program ceiling.
Replacement templates support $0, $1, ${1}, $name, ${name}, and
$$. Missing or unmatched captures expand to empty text. Unbraced names are
maximal: $1x denotes the capture named 1x, while ${1}x denotes capture
one followed by x. This distinction is covered by the pinned oracle.
Strings, lowercase booleans, numbers, and compact JSON arrays use the same
textual projection as literal replace; missing fields, null, and exact
object parents project to empty text. A no-match operation preserves the
original native value. An actual replacement writes a string only to the
request-owned query row, so durable rich metadata remains unchanged.
Sequential transformations see prior results.
Parsing, pattern compilation, captures, replacement expansion, projected arrays, output sizing, field paths, work, result rows, response bytes, and cancellation are bounded. Invalid patterns, attached syntax, wrong arity, wildcard targets, invalid or leading-zero limits, and trailing tokens fail explicitly.
There is no executable SQL-equivalent recipe for this row. Core SQLite and
the public timeless-libsql extension expose no portable RE2-compatible
replacement function with capture-template expansion. Claiming ordinary SQL
support would therefore be false; applications using SQLite directly must
compose this transformation outside SQL or deliberately load a separate
regexp extension. The existing public logs scan remains the storage
boundary, and measurements decide whether a future general-purpose extension
primitive is justified. Promoting LogsQL syntax or a language-specific
replacement helper into the storage extension is not justified.
Exact-build regexp-replacement evidence measures 3.303/3.442/3.533 ms
narrow and 39.620/40.628/45.369 ms wide p50/p95/p99 while returning 64 rows
and 1,600 response bytes. Byte-identical same-scan controls measure
3.200/3.391/3.646 and 33.545/35.822/36.126 ms. The +1.5%/+13.4% p95 and
+0.2%/+19.8% internal API cost follows the same one/four candidate blocks,
1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and 128/8,192
public rows. QSF-181 accepts the bounded API-side regex and capture-
expansion work above the unchanged public storage boundary.
extract captures text between fixed literal delimiters into named fields:
* | extract 'kind=<kind> id=<id>'
* | extract '<left> < <right>' from comparison
* | extract 'ip=<ip> <_>=<method> path=<path>' from request
* | extract if (service:=api) 'user=<user>' keep_original_fields
* | extract 'value=<plain:raw_value>' from payload skip_empty_results
The command and the if, from, keep_original_fields, and
skip_empty_results keywords are case-insensitive. The quoted or unquoted
pattern must contain at least one named <field>. <>, <_>, and <*> are
anonymous captures. Adjacent placeholders are invalid: every pair needs a
nonempty literal delimiter. Literal pattern text is HTML-decoded, so <
matches <. The source defaults to _msg and may be one exact quoted or
dotted current-row field. A nonempty first literal may begin anywhere in the
source; an empty first literal anchors extraction at the start.
When a capture begins with a valid Go double-quoted, single-quoted, or raw
backtick string, extract decodes that quoted prefix and then requires the
next literal immediately after it. The plain: field option disables this
automatic decoding. A missing first literal leaves every named result empty.
If a later unquoted delimiter is missing, earlier completed fields remain and
the current/later fields are empty. A successfully decoded quoted field also
remains available when its following delimiter is missing. Explicit empty
result strings remain present in Timeless output rather than becoming
indistinguishable from missing metadata.
By default every named capture replaces its current-row destination, including
an empty capture. keep_original_fields preserves each destination whose
existing textual value is nonempty. skip_empty_results preserves a nonempty
existing destination only when its new capture is empty; nonempty captures
still replace it. Existing numbers, booleans, arrays, and objects count as
nonempty and remain native whenever preserved. Source strings, lowercase
booleans, numbers, and compact arrays use the established textual projection;
missing, null, and exact object parents project to empty text. A capture may
write a nested leaf while preserving its siblings, but replacing a retained
object with a scalar fails explicitly with 422. All transformations are
request-local; durable log metadata is unchanged, and later pipeline stages
observe earlier results.
Pattern traversal, quoted decoding, projected arrays, captures, destination paths, work, result rows, response bytes, and cancellation use the shared hard request limits. Missing/literal-only patterns, wildcard sources or outputs, adjacent fields, misplaced conditions, both preservation modifiers, malformed quotes, and trailing tokens fail instead of being ignored.
Executable
SQL-LOG-040
uses only public logs rows, SQLite JSON1, and core instr()/substr() to
extract two unquoted fields from a fixed prefix/middle/suffix pattern. General
pattern parsing, Go quoted-string decoding, current-row mutation and preserve
modes, limits, cancellation, and HTTP envelopes remain Rust API behavior. No
extension primitive, private table, storage-format change, or durable
mutation is involved.
Exact-build literal-extraction evidence measures 2.977/3.269/3.673 ms narrow
and 37.121/39.052/43.064 ms wide p50/p95/p99 while returning 64 rows and
1,600 response bytes. Byte-identical same-scan controls measure
2.925/3.201/3.315 and 32.704/33.944/34.306 ms. The +2.1%/+15.0% p95 and
-1.7%/+12.2% internal API variation follows the same one/four candidate
blocks, 1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and
128/8,192 public rows. QSF-183 accepts this bounded API-side literal scan,
quoted decoding, and field-write work above the unchanged public storage
boundary.
extract_regexp writes the named captures from the first regular-expression
match into request-owned fields:
* | extract_regexp 'user=(?P<user>[A-Za-z]+) id=([0-9]+)'
* | extract_regexp 'kind=(?<kind>[a-z]+)' from payload
* | extract_regexp if (service:=api) 'request=(?P<request>.+)' keep_original_fields
* | extract_regexp '(?P<line>.+)' from "source field" skip_empty_results
The command and the if, from, keep_original_fields, and
skip_empty_results keywords are case-insensitive. The quoted or unquoted
RE2-family pattern must contain at least one named group. Both
(?P<name>...) and (?<name>...) are accepted. Anonymous groups affect the
match but create no field. Backreferences and lookaround are rejected. The
source defaults to _msg and may be one exact quoted or dotted current-row
field; wildcard sources and capture destinations are rejected.
Only the first match is used. Dot matches newline by default, matching
VictoriaLogs; inline flags such as (?-s) may disable that behavior. A
missing match or unmatched optional named group produces an empty capture.
Default mode writes that empty string, keep_original_fields preserves every
nonempty existing destination, and skip_empty_results preserves a nonempty
destination only when the new capture is empty. Later stages observe earlier
writes. Strings, lowercase booleans, numbers, and compact JSON arrays use the
standard textual projection. Preserved native numbers, booleans, arrays,
objects, nulls, and nested siblings are not rewritten. Replacing a retained
object with a scalar fails with HTTP 422 instead of silently discarding it.
Regex compilation is request-once and size-bounded. Source projection, captures, paths, work, temporary state, result rows, response bytes, deadlines, and cancellation use the shared hard limits. Transformations never mutate durable log metadata and survive flush, optimize, shutdown, and reopen through the same public storage rows.
There is intentionally no SQL-equivalent recipe for this row. Core SQLite and the public timeless-libsql extension do not provide a portable RE2-compatible named-capture extraction scalar. Direct users can apply a host regex or load a separate general regexp extension; timeless-libsql does not claim that as ordinary SQL support. No private table, extension storage primitive, or durable-format change is involved.
Exact-build regexp-extraction evidence measures 3.027/3.154/4.780 ms narrow
and 34.890/35.517/37.085 ms wide p50/p95/p99 while returning 64 rows and
1,600 response bytes. Byte-identical same-scan controls measure
3.034/3.922/4.296 and 32.696/33.808/38.479 ms. The -19.6%/+5.1% p95 and
-1.6%/+6.1% internal API variation follows the same one/four candidate
blocks, 1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and
128/8,192 public rows. QSF-185 accepts the bounded API-side first-match
capture and field-write work above the unchanged public storage boundary.
pack_json snapshots selected request-owned fields and writes one compact
JSON string to a current-row destination:
* | pack_json
* | pack_json as packed
* | pack_json packed
* | pack_json fields (host, status, context.*) as packed
* | pack_json fields ("request."*) as request_json
The command, fields, and as are case-insensitive. The destination defaults
to _msg; explicit destinations may follow as or be bare. Omitted or empty
fields (...) selects all fields, as does * anywhere in the list. Exact and
prefix selectors may be quoted. Selection snapshots the row before the
destination write: pack_json includes the old _msg inside the new _msg,
and packing over an existing destination captures its old value. Later stages
observe the packed string. Missing exact fields yield {}; overlapping
selectors form one idempotent union in deterministic key order.
Timeless preserves the retained JSON model rather than flattening it. Numbers, booleans, arrays, objects, explicit nulls, empty strings, and empty objects retain their native JSON representation, while dotted prefix selection reconstructs nested objects. This intentionally differs from VictoriaLogs v1.52.0, which flattens current columns to strings, omits empty values, follows column order, and can emit duplicate keys for overlapping selectors. The pinned 850-case oracle records the upstream behavior; real- extension regressions pin the richer Timeless compatibility policy.
Paths, recursive visits, selected values, temporary JSON bytes, nesting,
work, result rows, response bytes, deadlines, and cancellation are bounded by
the shared request limits. Destinations under scalar parents fail with HTTP
422. Packing is query-local and never changes the public durable logs rows,
including after optimize, shutdown, or reopen.
Executable
SQL-LOG-041
uses public logs, json_type, ->, and json_set to pack a bounded list
of exact JSON paths while preserving missing/null/empty/type distinctions.
Recursive prefix/all selectors, destination writes, language errors, limits,
cancellation, and HTTP envelopes remain Rust API composition. No extension
primitive, private table, or storage-format change is involved.
Exact-build typed-packing evidence measures 2.946/3.146/4.588 ms narrow and
35.570/37.921/38.085 ms wide p50/p95/p99 while returning 64 rows and 2,688
response bytes. Same-scan plain-field controls measure
2.959/3.098/3.227 and 32.567/35.717/37.101 ms while returning 1,600 bytes.
The +1.5%/+6.2% p95 and -1.3%/+7.4% internal API variation follows the same
one/four candidate blocks, 1,024/8,192 decoded entries,
235,778/1,914,055 payload bytes, and 128/8,192 public rows. QSF-187
accepts the bounded rich selection and serialization cost above the unchanged
public storage boundary.
pack_logfmt snapshots selected request-owned fields and writes one logfmt
string to a current-row destination:
* | pack_logfmt
* | pack_logfmt as packed
* | pack_logfmt packed
* | pack_logfmt fields (host, status, context.*) as packed
* | pack_logfmt fields (missing, *,) as "packed field"
The command, fields, and as are case-insensitive. The destination defaults
to _msg; it may follow as or be bare, and terminal as keeps the default.
Omitted or empty fields (...) selects all current fields, as does *
anywhere in the list. Exact and suffix-wildcard prefix selectors may be
quoted. Selection snapshots the row before writing, so replacing _msg or an
existing destination includes its prior value. Later stages observe the new
text.
Output is a space-separated sequence of raw name=value pairs in
deterministic bytewise field-name order. Missing exact fields, explicit null,
empty strings, and exact object parents emit empty values. All/prefix
selection recursively flattens objects to dotted leaves; arrays stay atomic
compact JSON. A value is quoted exactly when it contains a rune through
U+0020, a double quote, or a backslash. Quoted values use the pinned
VictoriaLogs JSON spelling, including \u003c and \u0027; otherwise they
remain unquoted.
VictoriaLogs v1.52.0 preserves current column order and repeats fields for overlapping selectors. Timeless intentionally forms an idempotent selector union and orders retained field names deterministically. This avoids unstable duplicates while preserving the richer nested model. The 1,111-case pinned oracle records upstream behavior, and the real-extension regression pins the selected retained-model policy through optimize and reopen.
Recursive traversal, names, textual projections, output bytes, work, result
rows, response bytes, deadlines, and cancellation are bounded. A destination
that would replace an object or descend through a scalar fails with HTTP 422.
The operation mutates only request-owned rows, never public durable logs.
Executable
SQL-LOG-052
uses public logs, JSON1, json_quote, and deterministic aggregation for a
fixed ordered list of exact metadata paths. Dynamic selectors, canonical
fields, current-row writes, errors, limits, cancellation, and HTTP envelopes
remain Rust API composition. No extension primitive, private table, or
storage-format change is involved.
Exact-build logfmt evidence measures 3.459/3.805/4.335 ms narrow and
37.975/39.144/42.387 ms wide p50/p95/p99 while returning 64 rows and 2,048
bytes. Identical-output format controls measure 3.506/3.769/3.924 and
34.373/38.212/38.572 ms. The +1.0%/+2.4% p95 and -0.2%/+8.2% internal API
mean follows exactly the same one/four candidate blocks, 1,024/8,192 decoded
entries, 235,778/1,914,055 payload bytes, and 128/8,192 public rows.
QSF-218 accepts the bounded dynamic selection/encoding cost above the
unchanged public storage boundary.
unpack_json snapshots one request-owned field, parses a JSON object, and
writes selected members back into the current result row:
* | unpack_json
* | unpack_json from payload
* | unpack_json payload fields (host, status, context.*)
* | unpack_json if (kind:=audit) from payload preserve_keys (context)
result_prefix decoded. keep_original_fields
* | unpack_json from payload fields () skip_empty_results
Keywords are case-insensitive. The source defaults to _msg; one exact
source may appear bare or after from. The source may be whitespace-padded
JSON-object text or a retained native object. Omitted or empty fields ()
selects all fields. Exact and prefix selectors may be mixed; missing exact
paths become empty strings, while unmatched prefixes produce no fields.
preserve_keys keeps named objects atomic and native. result_prefix is
prepended before reconstructed output paths.
Timeless preserves the retained rich JSON model. Strings, numbers, booleans,
arrays, objects, explicit nulls, empty strings, and empty objects retain their
native types. Nested output merges with unrelated existing siblings, and a
literal JSON key containing . remains distinct from a nested path. The
source is snapshotted before any destination write, including when an
unpacked member replaces the source itself. By default selected values
overwrite scalar destinations. keep_original_fields retains existing
nonempty values; skip_empty_results suppresses incoming null and empty
strings. Writes through a scalar parent or scalar replacement of a retained
object fail with HTTP 422.
Whitespace is ignored around object text. Missing, null, scalar, array, and
nonobject strings are no-ops. For compatibility, malformed text beginning
with { writes empty strings only for explicitly requested exact paths, and
the pinned bare NaN token becomes the string "NaN". Grammar, JSON/path
work, parsed and selected state, result rows, response bytes, deadlines, and
cancellation are bounded by shared request limits. The transform never
changes durable public rows, including after optimize, shutdown, and reopen.
VictoriaLogs v1.52.0 flattens nested values to textual columns, serializes arrays compactly, textualizes numbers/booleans, and maps null to empty text. The pinned 875-case oracle records that language behavior; Timeless's native types and reconstructed nesting are the documented retained-model policy.
Executable
SQL-LOG-042
uses public logs, json_valid, json_type, ->, and json_set for a
bounded fixed set of exact paths while preserving missing/null/empty/type
distinctions. Dynamic selectors, request-local mutation and preservation,
language errors, limits, cancellation, and envelopes remain Rust API
composition. No extension primitive, private table, or storage-format change
is involved.
Exact-build typed-unpacking evidence measures 2.933/3.151/3.256 ms narrow and
36.963/40.062/65.292 ms wide p50/p95/p99 while returning 64 rows and 2,112
response bytes. Equal-output pack-plus-copy controls measure
2.962/3.763/4.407 and 36.314/38.694/41.375 ms. The -16.3%/+3.5% p95 and
-0.5%/+3.3% internal API variation follows the same one/four candidate
blocks, 1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and
128/8,192 public rows. QSF-189 accepts the bounded parse/select/write cost
above the unchanged public storage boundary and retains the wide p99 without
hiding it.
unpack_logfmt snapshots one request-owned field, parses logfmt, and writes
selected string values back into the current result row:
* | unpack_logfmt
* | unpack_logfmt from payload
* | unpack_logfmt payload fields (host, status, context.*)
* | unpack_logfmt if (kind:=audit) from payload
fields (host, context.*) result_prefix decoded. keep_original_fields
* | unpack_logfmt from payload fields () skip_empty_results
Keywords are case-insensitive. The source defaults to _msg; one exact
source may appear bare or after from. Omitted or empty fields () selects
all parsed names. Exact and prefix selectors may be mixed. Missing exact
names become empty strings, while unmatched prefixes add nothing.
result_prefix is prepended to every selected name.
Unquoted values end at ASCII space. Double- and single-quoted values decode
Go-compatible control, quote, backslash, octal, \x, \u, and \U
escapes. Backtick-quoted values are raw and discard carriage returns. Invalid
or unterminated quoted prefixes fall back to unquoted parsing, matching the
pinned VictoriaLogs behavior. Lone names become empty values, duplicate names
are last-wins, and every decoded value is a string.
Timeless reconstructs dotted decoded names and prefixes as nested metadata
while preserving unrelated siblings. A name containing an empty path segment
stays a literal top-level key rather than creating an invalid nested path.
The source is snapshotted before writes, including when one decoded name
replaces it. By default decoded strings replace scalar destinations.
keep_original_fields retains an existing nonempty destination;
skip_empty_results suppresses decoded empty strings. A write that would
replace a retained object or descend through a scalar returns HTTP 422.
Strings, projected native numbers/booleans/arrays, names, paths, decoded
bytes, work, temporary state, result rows, response bytes, deadlines, and
cancellation use the shared hard limits. The transform changes only
request-owned rows, never the public durable logs source, including after
optimize, shutdown, and reopen. The complete 1,134-case pinned VictoriaLogs
fixture records upstream grammar and value behavior; real-extension tests pin
Timeless's retained nesting policy.
Executable
SQL-LOG-053
uses a bounded recursive CTE over public logs to extract a fixed set of keys
from well-formed unquoted logfmt. Full quoted/escaped parsing, dynamic
selectors, current-row writes, errors, limits, cancellation, and envelopes
remain Rust API composition. No extension primitive, private table, or
storage-format change is involved.
Exact-build logfmt-unpacking evidence measures 3.355/3.619/4.494 ms narrow
and 41.166/42.437/43.357 ms wide p50/p95/p99 while returning 64 rows and
2,112 bytes. Identical-output pack-plus-copy controls measure
3.328/3.573/4.213 and 38.087/41.659/44.310 ms. The +1.3%/+1.9% p95 and
+1.2%/+5.9% internal API mean follow the same one/four candidate blocks,
1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and
128/8,192 public rows. QSF-220 accepts the bounded quoted/unquoted parse,
selection, and nested destination work above the unchanged public storage
boundary.
unpack_syslog snapshots one request-owned field, parses syslog, and writes
decoded string fields back into the current result row:
* | unpack_syslog
* | unpack_syslog from payload
* | unpack_syslog payload offset -5h result_prefix decoded.
* | unpack_syslog if (kind:=audit) from payload
offset 30m result_prefix decoded. keep_original_fields
Keywords are case-insensitive. Clauses must remain in the displayed order:
optional if (...), optional bare or from exact source, optional signed
duration offset, optional result_prefix, then optional terminal
keep_original_fields. The source defaults to _msg; missing sources are
no-ops. Only leading ASCII spaces, tabs, carriage returns, and newlines are
trimmed. Sources are snapshotted before writes, so decoding into a source or
prefixed sibling is deterministic.
The parser accepts RFC3164 and RFC5424 with optional PRI. It emits textual
priority, facility_keyword, level, facility, severity, format,
timestamp, hostname, app_name, proc_id, msg_id, and message when
present. RFC5424 timestamps retain their lexical spelling and structured data
becomes SD-ID.parameter. Classic RFC3164 timestamps use the request year and
host timezone unless offset supplies a fixed offset; a value more than one
day in the future moves to the previous year. Leap-day normalization matches
Go time.Date. RFC3164 messages carrying RFC3339/ISO timestamps normalize to
UTC and do not use the classic-time offset.
CEF messages produce cef.version, device fields, severity, and
cef.extension.*. CEE JSON objects become decoded fields: strings remain
text, numbers and booleans become text, arrays become compact JSON text,
nested objects flatten to dotted names, and null members are omitted. Invalid
PRI and CEF inputs retain VictoriaLogs' partial/fallback behavior rather than
being silently reinterpreted.
Timeless reconstructs dotted decoded fields and result prefixes as retained
nested metadata while preserving unrelated siblings. This is the explicit
richer-model compatibility policy over VictoriaLogs' flattened textual rows.
Default writes replace scalar destinations; keep_original_fields retains an
existing nonempty destination. Replacing a retained object or descending
through a scalar returns HTTP 422. Source bytes, decoded names/values, nesting,
paths, temporary state, work, results, response bytes, deadlines, and
cancellation use shared hard limits. Query-backed conditions share those
limits. Request-local writes never mutate durable logs rows through
optimize, shutdown, or reopen.
The complete 1,155-case immutable VictoriaLogs fixture records grammar,
header/structured parsing, CEF/CEE, conditions, prefixes, preservation,
partial input, and errors. Executable
SQL-LOG-054
uses only bounded public logs rows and core SQLite for the fixed RFC5424
header when structured data is -. RFC3164, structured data, CEF/CEE,
timezone/year behavior, current-row mutation, limits, cancellation, and
envelopes remain Rust API composition. No extension primitive, private table,
or storage-format change is involved.
Exact-build syslog-unpacking evidence measures 3.391/3.581/3.802 ms narrow
and 33.887/38.081/38.210 ms wide p50/p95/p99 while returning 64 rows and
1,984 bytes. Identical-output format-plus-copy controls measure
2.978/3.439/3.584 and 34.075/38.026/38.209 ms. The +4.1%/+0.1% p95 and
+9.5%/-0.8% internal API mean follow the same one/four candidate blocks,
1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and
128/8,192 public rows. The benchmark sorts/materializes those public rows and
limits to 64 before expansion. QSF-223 records that expanding all 8,192
formatted rows correctly exceeds the default work budget; configure more work
or narrow/limit first. QSF-224 accepts the bounded returned-row parse above
the unchanged storage boundary.
unpack_words snapshots one exact request-owned field, extracts ordered word
tokens, and writes compact JSON-array text to the current result row:
* | unpack_words
* | unpack_words from payload
* | unpack_words payload words
* | unpack_words from payload as words drop_duplicates
* | unpack_words as "word list"
Keywords are case-insensitive. The exact source defaults to _msg; the exact
destination defaults to the source. Bare and explicit from/as forms,
quoted fields, and dotted paths are accepted. The source is read before the
destination is written. A terminal drop_duplicates keeps the first
byte-identical occurrence in source order. Wildcards, prefixes,
comma-separated fields, attached suffixes, and trailing tokens fail before
storage execution.
A word is one maximal sequence of Unicode Letter, Unicode Decimal_Number, or
underscore characters. Unicode Letter_Number, Other_Number, combining marks,
punctuation, and whitespace split words. Case and normalization are not
changed. String sources are exact; numbers and booleans use compact text;
arrays use compact JSON text; missing, null, and object-parent sources project
as empty. Empty and punctuation-only sources produce [].
Timeless writes dotted destinations into retained nested metadata without discarding siblings. A destination that would replace an object or descend through a scalar fails with HTTP 422. Input characters, tokens, duplicate state, output bytes, paths, work, results, response bytes, deadlines, and cancellation are bounded. Request-local writes never mutate public durable rows through optimize, shutdown, or reopen. The complete 1,175-case immutable VictoriaLogs fixture and real-extension regression pin this behavior.
There is deliberately no SQL recipe for complete LQL-P39. Core
SQLite/libSQL cannot portably implement the exact Unicode category rule;
ASCII recursive SQL would silently change results. Every candidate row must
already cross the bounded public logs interface, so a new extension scalar
would not avoid block reads, decode, payload transfer, or required response
materialization. The Rust logs API owns this transform without a private table,
language opcode, or storage-format change.
Exact-build p50/p95/p99 is 3.225/3.740/4.877 ms narrow and
36.912/38.564/39.308 ms wide, versus 3.250/3.493/3.534 and
34.241/35.662/36.075 ms for equal-storage-work copies. The +7.0%/+8.1% p95
includes Unicode classification, deduplication, compact-array creation, and a
640-byte larger response after byte-identical public storage work.
QSF-227 accepts the bounded Rust API cost and preserves the no-extension
verdict.
json_array_concat snapshots one exact request-owned field, joins its
top-level array elements with an optional delimiter, and writes TEXT to one
exact current-row destination:
* | json_array_concat
* | json_array_concat ","
* | json_array_concat from tags as joined
* | json_array_concat " | " tags joined
* | json_array_concat "\n" from payload.items
Keywords are case-insensitive. Source and destination default to _msg and
the source respectively. from and as are optional when the operands are
unambiguous; fields may be quoted or dotted. The delimiter is a decoded quoted
value or one supported compound token. Wildcards, prefixes, attached
suffixes, missing operands, and trailing tokens fail before storage work.
Retained native arrays are traversed without changing their types. String
sources accept valid JSON arrays with surrounding JSON whitespace. Top-level
strings are decoded and joined without JSON quotes; nulls, booleans, numbers,
objects, and nested arrays use compact JSON. The pinned VictoriaLogs bare
NaN token is retained. For string-backed arrays, the bounded scanner also
preserves raw numeric spelling (1.00, -0, 1e3), object key order, and
nested escape spelling while removing only insignificant JSON whitespace.
Empty arrays, missing/null sources, malformed or scalar JSON text, and native
nonarrays produce an explicit empty string.
VictoriaLogs internally assigns that empty result but its streaming JSON
encoder omits empty-valued columns. Timeless returns "" so its retained
missing-versus-null-versus-empty contract remains observable. This explicit
response-encoding difference preserves more fidelity without changing the
transform value.
The source is fully read before any destination write. Dotted destinations reconstruct nesting and preserve siblings; a write that would replace an object or descend through a scalar fails with HTTP 422. JSON depth, token work, decoded strings, span state, result bytes, rows, response bytes, deadlines, and cancellation are bounded. Durable public rows remain unchanged through optimize, shutdown, and reopen.
Executable
SQL-LOG-055
uses only bounded public logs rows plus SQLite JSON1 for a fixed canonical
array path. SQLite cannot preserve raw numeric spellings or bare NaN, so the
complete LogsQL transform remains in the Rust logs API. No extension opcode,
private shadow table, storage format, or batching behavior is added.
Exact-build p50/p95/p99 is 3.202/3.872/4.930 ms narrow and
34.373/35.216/35.316 ms wide, versus 3.332/3.418/3.614 and
38.555/40.200/40.598 ms for equal-output format controls. The +13.3%/-12.4%
p95 and -1.1%/-8.4% request-attributed API mean follow byte-identical public
storage work and identical 64-row, 1,536-byte responses. QSF-229 retains the
opposing narrow tail/mean honestly and accepts the bounded post-scan cost.
json_array_len snapshots one exact request-owned field, counts its top-level
array elements, and writes a decimal string to the current result row:
* | json_array_len(tags)
* | json_array_len(tags) as tag_count
* | json_array_len payload.items item_count
* | json_array_len("left field") as "item count"
Keywords are case-insensitive. Parenthesized and bare exact sources are
accepted; as is optional and a terminal as retains the default _msg
destination. Sources and destinations may be quoted or dotted exact paths.
Wildcards, prefixes, multiple sources, and trailing tokens fail explicitly.
The source is read before the destination is written, so
json_array_len(tags) as tags deterministically replaces the request-local
field with its count without changing storage.
Retained native arrays are counted directly. Strings containing valid JSON
arrays may have surrounding whitespace and are parsed; the pinned
VictoriaLogs bare NaN token counts as one element. Nested arrays and objects
each count once. Empty arrays, missing fields, explicit nulls, malformed JSON,
JSON scalar text, and native scalar or object values return "0". Native
source arrays and all of their element types remain unchanged. Writes that
would replace a retained object fail with HTTP 422. Parsing, temporary state,
paths, rows, response bytes, deadlines, and cancellation all use the shared
request limits, and the reader remains reusable after rejection.
The pinned VictoriaLogs v1.52.0 oracle covers the command's grammar, array, scalar, malformed, default-destination, overwrite, case, and error behavior in the complete 897-case fixture. The real-extension regression additionally pins Timeless's native rich arrays, microsecond durability, optimize, shutdown, and reopen.
Executable
SQL-LOG-043
uses public logs, json_type, json_valid, and json_array_length for a
bounded fixed exact path. It returns textual zero for nonarrays and leaves the
source untouched. Language grammar, current-row destination writes,
VictoriaLogs bare-NaN compatibility, limits, cancellation, and envelopes
remain Rust API composition. No extension primitive or private storage access
is involved.
Exact-build native-array evidence measures 3.285/3.558/3.788 ms narrow and
39.914/41.563/44.668 ms wide p50/p95/p99 while returning 64 rows and 1,344
response bytes. Equal-output constant-format controls measure
3.276/3.454/3.514 and 39.836/40.607/43.555 ms. The +3.0%/+2.4% p95 and
+2.4%/-0.0% internal API variation follows the same one/four candidate
blocks, 1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and
128/8,192 public rows. QSF-191 accepts the bounded direct native-array count
and request-local write above the unchanged public storage boundary.
unroll snapshots one or more exact request-owned fields and expands their
top-level array elements into current result rows:
* | unroll tags
* | unroll source, other
* | unroll by (source, other)
* | unroll if (kind:="expand") (payload.items, "left field")
Keywords are case-insensitive. if and by are optional; fields may be bare
or parenthesized, quoted, or dotted. Parenthesized lists accept a terminal
comma. Empty lists, wildcards, prefixes, unparenthesized trailing commas,
conditions after the field list, attached suffixes, and trailing tokens fail
before storage work.
All selected values are read before the first write. Multiple arrays zip by
index and emit as many rows as the longest source. Shorter, missing, null,
malformed, scalar, and native nonarray sources contribute ""; if no source
has an element, exactly one empty-valued row is emitted. A false condition
passes the complete rich row through once without rewriting it. Dotted writes
preserve siblings; overlapping selected paths or a scalar parent return HTTP
422 instead of producing order-dependent output.
Native retained arrays are traversed without flattening. String sources accept
JSON arrays with surrounding whitespace. Top-level strings are decoded;
nonstrings become compact JSON text. Raw number spellings (1.00, -0,
1e3), object order, and bare NaN match VictoriaLogs. Unlike
json_array_concat, nested JSON string escapes are decoded and re-encoded, so
{"a":"\\u0061"} becomes {"a":"a"}. Timeless returns explicit empty
strings where the VictoriaLogs streaming encoder omits empty-valued columns,
preserving Timeless's missing/null/empty response model.
Input and output rows, source snapshots, raw scans, normalized strings, result cardinality, work, response bytes, deadlines, query-backed predicates, and cancellation are bounded cumulatively. The reader remains reusable after rejection. Request-local expansion never mutates public log rows, including after optimize, shutdown, and reopen.
Executable
SQL-LOG-056
uses only bounded public logs rows and JSON1 to expand one fixed canonical
array. SQLite cannot preserve every raw numeric/NaN spelling, and ordinary
SQL does not supply multi-field rich-row zip mutation or API envelopes. Those
complete semantics stay in bounded Rust composition after the same public
scan. No extension primitive, private table, durable format, compression,
index, or authoritative 8,192-entry batch contract changes.
Exact-build p50/p95/p99 is 3.552/3.896/4.049 ms narrow and
35.071/39.121/40.075 ms wide while returning 128 rows and 2,112 bytes. The
array-concat controls measure 3.220/3.670/4.680 and
34.595/39.180/39.443 ms while returning 64 rows and 1,408 bytes. QSF-231
accepts the +6.1%/-0.2% p95 and +8.3%/-0.0% request-attributed API mean after
identical public block, decode, payload, and row work. The candidate's doubled
cardinality and 704 additional bytes are retained in the endpoint result; no
independent equal-cardinality expansion control exists at this boundary.
join combines each current left row with inline or query-backed right rows
using one or more exact textual keys:
* | join by (service) (kind:="deployment" | fields service, owner)
* | join on (service, region) (kind:="deployment" | fields service, region, owner) inner
* | join by (service) rows({service:"api",owner:"platform"}) prefix deployment.
by and on are interchangeable and case-insensitive. Keys must be a
nonempty parenthesized list of exact bare, quoted, or dotted fields; wildcards
and prefixes fail. The right source is either a parenthesized LogsQL query or
rows({field:value,...},...). Inline names and scalar compound values may be
quoted, : and = are accepted, terminal commas are allowed, and arrays or
objects are rejected as inline values. Quoted dotted names are literal;
unquoted dotted names address retained nested paths. The optional inner and
prefix value modifiers may appear as inner prefix or prefix inner, once
each. Unsupported or malformed syntax fails before storage work.
The default is a left join: an unmatched row passes through unchanged. inner
drops it. Missing, explicit null, and empty values share the empty textual key;
numbers, booleans, arrays, objects, and strings use the same public rich-row
text projection as other LogsQL operations. Every matching right row emits an
output in right-source order, so duplicate keys expand cardinality. Left order
is retained. Use an explicit supported sort inside either query when that
order matters.
All join-key fields are removed from the right payload. With no prefix, a
nonempty left value wins a collision; a missing, null, or empty left value is
filled. Prefixes are prepended before retained nested insertion. Timeless
preserves right strings, numbers, booleans, arrays, objects, explicit nulls,
and nested paths instead of flattening everything to VictoriaLogs strings.
Unrelated left siblings remain intact, while a required object path through a
scalar parent returns HTTP 422 field_conflict. Joins are request-local and
never modify either public source row.
Nested right queries run through the same Rust parser and public logs
interface, share one request clock, recursively resolve their own query-backed
operations, and inherit parent work, result, response, deadline, and nesting
limits. Inline and query rows, their rich values, textual keys, duplicate
indexes, output rows, and response bytes are all bounded; cancellation is
checked during map construction, matching, recursive rich merging, and
output. Empty rows() is a left identity and produces no inner results.
The complete 1,249-case pinned VictoriaLogs fixture covers twelve successful
join vectors and fourteen strict failures. Real-extension regressions add
typed/nested fidelity, numeric/string key equivalence, collision envelopes,
limits, immutable source data, optimize, shutdown, and reopen. Executable
SQL-LOG-057
provides the direct SQLite/libSQL foundation: two independently bounded public
scans, textual key projection, deterministic duplicate expansion, optional
inner filtering, and separate typed left/right payloads. The API owns LogsQL
grammar and rich merge policy. Both scans and payload crossings are required,
so no language-specific extension primitive or private storage access is
justified.
Exact release build 8c718dceccfb56f251f7f54084976cc69233de7e
measures a 64-row query-backed join at 6.568/6.875/7.299 ms narrow and
66.254/73.000/73.984 ms wide p50/p95/p99. The independent query-backed
in(...) controls return the same 64 rows and 1,280 bytes at
5.934/6.225/6.666 and 65.809/72.087/72.452 ms. Join p95 is therefore 10.4%
higher narrow and 1.3% higher wide; request-attributed API work is 14.7% and
0.5% higher. Each request executes exactly two public scans. Candidate and
control read the same one/four blocks per scan, decode 1,024/8,192 entries,
read 235,778/1,914,055 payload bytes, and cross 128/8,192 public rows per
scan. Join explicitly reserves 192 additional work items for its 64 retained
two-field right rows, so its outer requested ceiling is lower without
changing physical storage work. The bounded RHS materialization, index, and
rich merge cost is accepted above the public storage boundary; it does not
justify moving LogsQL syntax or a join map into the extension.
LogsQL union appends inline rows or the result of another complete LogsQL
pipeline to the current request-owned rows:
kind:="request" | union (kind:="deployment" | fields service, owner)
kind:="request" | union rows({service:"api",owner:"platform"},{service:"worker",owner:"jobs"})
kind:="request" | union (kind:="deployment" | union rows({service:"local"})) | stats count() as rows
union and rows are case-insensitive. A source is either one parenthesized
query or strict rows(...); inline fields accept : or =, quoted compound
tokens, optional commas, and dotted retained paths, but values are textual
scalars. Arrays, objects, missing values, malformed rows, empty subqueries,
trailing tokens, and attached suffixes fail before storage work. rows() is
an identity. An inline {} carries no fields and adds no result row, matching
the pinned upstream response. Duplicates are preserved.
Query-backed sources recursively use the same parser, request clock, public
logs table, and parent limits. Timeless preserves their strings, numbers,
booleans, arrays, objects, explicit nulls, empty strings, and nested paths.
It appends all left rows in their current order and then all source rows in
their current order; later filters, fields, sorts, limits, and statistics see
the combined sequence. VictoriaLogs' live multi-worker HTTP response can
reorder rows without a later sort, despite its processor's append-at-flush
design, so portable LogsQL callers should still sort whenever order is part
of the contract.
Both scans, retained rich source state, cloned output rows, recursive value traversal, nesting/count, result cardinality, response bytes, deadline, and cancellation are bounded cumulatively. Composition is request-local and leaves durable rows unchanged through optimize, shutdown, and reopen. The complete 1,267-case pinned VictoriaLogs fixture covers nine successful and nine strict error vectors; the real-extension regression adds retained rich fidelity, deterministic ordering, cumulative limits, immutability, and durability.
Executable
SQL-LOG-058
gives direct SQLite/libSQL users the ordinary UNION ALL foundation over two
independently bounded public scans with explicit source order and complete
typed payloads. Inline sources are ordinary bounded VALUES CTEs. The API
owns LogsQL grammar, recursive planning, limits, cancellation, and envelopes.
No extension primitive, private table, storage format, or authoritative
8,192-entry batching behavior changes.
Exact release build c3ce658ed9475691fc8e4a51e9fe0e02fb57fe7c
measures query-backed union at 6.442/6.794/6.917 ms narrow and
67.816/74.143/75.364 ms wide p50/p95/p99. Equal-output two-scan controls
measure 5.846/6.363/7.126 and 68.604/73.714/79.489 ms. Union p95 is
6.8%/0.6% higher; request-attributed API means are 5.370/67.574 ms versus
4.931/67.884 ms, or 8.9% higher/0.5% lower. Every request performs exactly
two public scans and returns the same 128 rows/2,560 bytes. Each candidate
and control reads the same one/four blocks per scan, decodes 1,024/8,192
entries, reads 235,778/1,914,055 payload bytes, and returns 128/8,192 public
rows per scan. The candidate deliberately reserves exactly 128 additional
work items per request for 64 retained one-member source rows; all physical
storage counters remain identical. The bounded post-scan cost is accepted
above the public storage boundary and provides no evidence for a new
extension primitive.
running_stats adds one or more cumulative fields while preserving input
cardinality:
* | running_stats count() as rows
* | running_stats by (service, level) sum(duration_ms) as elapsed
* | running_stats (service) min(duration_ms) as minimum, max(duration_ms) as maximum
* | running_stats first(message) offset 1 as second_message, last(message) offset 1 as previous_message
* | running_stats sum(payload.*) as nested_total
The command, grouping keyword, and function names are case-insensitive. The
optional group is written as by (field, ...) or (field, ...) and accepts
only nonempty exact fields. count, sum, min, and max accept exact,
prefix, or all-current-field selectors; omitted arguments mean all. first
and last require one exact field and accept a nonnegative offset. Each
function may use as result or a bare result name. Omitted names are
canonical, such as count(*) and sum(duration_ms). Duplicate or wildcard
destinations and malformed groups, arity, offsets, commas, suffixes, or
trailing tokens fail explicitly.
Timeless orders each group by parsed numeric microsecond time with stable
input-order ties, then emits groups in deterministic lexical key order.
VictoriaLogs sorts formatted _time strings internally; unequal RFC3339
fraction widths can therefore place 31 microseconds before 30 microseconds.
Timeless deliberately chooses chronological numeric order. The upstream
public contract does not promise cross-group order, so portable callers
should add an explicit later sort when they require a different final order.
count() counts rows. A selected-field count advances once when any selected
value is nonempty. sum accepts finite textual or native numbers and skips
missing, null, empty, nonnumeric, NaN, and infinity inputs. State is
"NaN" until the first finite number; any later finite overflow is rendered
as "+Inf" or "-Inf", because JSON has no nonfinite number. min and max
use the complete LogsQL natural textual order but retain the winning Timeless
value's native string, number, boolean, array, object, or null type. first
selects the fixed zero-based offset from the beginning; last selects the
offset behind the current row. Missing or not-yet-available first/last state
is an empty string.
Prefix and all-field selectors recursively traverse retained object leaves; arrays remain atomic rich values. Exact selectors can retain complete nested objects. Every expression reads the original current row before any result alias is written, so expressions in the same pipe cannot accidentally depend on one another. Results are visible to later pipes and deliberately overwrite an existing scalar destination. Descending through a scalar parent returns HTTP 422 rather than flattening or mutating durable metadata.
Sorting keys, groups, recursive traversal, accumulators, offset history,
generated values, result rows, response bytes, deadline, and cancellation are
bounded by the existing request limits. The operator uses one public logs
scan, never a private shadow table, and leaves source rows unchanged through
optimize, shutdown, and reopen. Its ordinary-SQL foundation is
SQL-LOG-059:
SQLite windows provide fixed-key numeric running state directly, while the
Rust API owns dynamic LogsQL grammar, natural ordering, complete rich values,
limits, cancellation, and envelopes. No extension primitive or storage-format
change is needed.
Exact release build 3f4ef107973f73361cfd90eff6e31ea53bd58f0c
measures grouped running count at 3.174/3.437/3.524 ms narrow and
44.922/48.000/49.893 ms wide p50/p95/p99. Same-scan time-sort/constant
controls measure 3.207/3.771/4.821 and 34.292/38.690/39.518 ms. Running
statistics p95 is 8.9% lower/24.1% higher; request-attributed API means are
2.467/44.051 ms versus 2.554/33.894 ms, or 3.4% lower/30.0% higher. Every
pair reads the identical one/four blocks, decodes 1,024/8,192 entries, reads
235,778/1,914,055 payload bytes, and materializes 128/8,192 public rows.
The 64-row candidate/control responses are 951/1,024 bytes because their
projected names and values differ. The accepted wide partition/sort/state
cost is bounded language composition after the same public scan; fixed-key
SQLite windows already serve direct users, so it does not justify an
extension primitive.
total_stats uses the same strict functions and grouping grammar as
running_stats, but computes the final state for each complete group before
writing that same state onto every row:
* | total_stats count() as rows
* | total_stats by (service, level) sum(duration_ms) as total_duration
* | total_stats (service) min(duration_ms) as minimum, max(duration_ms) as maximum
* | total_stats first(message) offset 1 as second_message, last(message) offset 1 as penultimate_message
* | total_stats sum(payload.*) as nested_total
The command, optional by (field, ...) or (field, ...) group, functions,
field selectors, offsets, aliases, canonical destination names, and explicit
syntax failures are identical to running_stats. Group fields and
first/last sources must be exact; count, sum, min, and max accept
exact, prefix, all-current-field, or omitted selectors. Duplicate or wildcard
destinations and malformed groups, arity, offsets, commas, suffixes, or
trailing tokens fail before the public scan is evaluated.
Each group is ordered by numeric microsecond time with stable input-order
ties, and groups are emitted in deterministic lexical key order. This retains
the documented Timeless correction for VictoriaLogs' variable-width
formatted-time ordering defect. first selects the fixed zero-based offset
from the beginning of the complete group; last selects the fixed offset
from its end. The chosen values, final counts, final sums, and final extrema
are then repeated on every row in the group. Empty input stays empty.
Count, finite textual/native sum, nonfinite rendering, complete natural-order
rich minima/maxima, recursive prefix traversal, atomic arrays, original-row
expression snapshots, later-pipe visibility, scalar overwrite, and
scalar-parent conflicts follow running_stats exactly. In particular, a
group with no finite sum input receives the explicit string "NaN" on every
row rather than an invalid JSON number.
Group keys, chronological sort state, recursive traversal, accumulators,
offset values, generated fields, result rows, response bytes, deadlines, and
cancellation are bounded by the existing request limits. Evaluation uses one
public logs scan, never a private shadow table, and leaves stored rows
unchanged through optimize, shutdown, and reopen. Executable
SQL-LOG-060
provides full-partition fixed-key SQLite windows for direct users. Rust owns
dynamic LogsQL grammar, natural/rich semantics, limits, cancellation, and
HTTP envelopes; no extension primitive or storage-format change is needed.
Exact release build 3bfdf2c843ebc221916d20b35a1df98d111e3eb9
measures grouped total count at 3.557/3.744/4.797 ms narrow and
44.051/47.706/49.552 ms wide p50/p95/p99. Same-scan chronological-sort
constant controls measure 3.280/3.618/3.805 and 33.905/34.716/35.739 ms.
Total statistics p95 is 3.5%/37.4% higher; request-attributed API means are
2.796/43.581 ms versus 2.575/32.879 ms, or 8.6%/32.6% higher. Every
pair reads the identical one/four blocks, decodes 1,024/8,192 entries, reads
235,778/1,914,055 payload bytes, and materializes 128/8,192 public rows.
Candidates and controls each return 64 rows. Candidate responses are
896/960 bytes versus 1,024/1,088 because count results are native numbers
while the math constant control uses quoted values. The bounded wide
complete-group prepass and rich snapshot writes occur after the same public
scan; fixed-key full-partition SQL windows already serve direct users, so the
measured cost does not justify an extension primitive.
time_add adds one VictoriaLogs duration to the default _time field or
one exact current-row field:
* | time_add 5m
* | time_add -1.5s at observed_at
* | time_add 500ns at nested.received_at
The command and optional at keyword are case-insensitive. Durations may be
negative and may combine decimal ns, µs, ms, s, m, h, d,
w, and 365-day y segments. A leading +, an unsuffixed number, a
missing duration or field, wildcards, prefixes, attached suffixes, and trailing
tokens fail explicitly before storage is queried.
Only a string accepted by the pinned VictoriaLogs RFC3339Nano parser changes.
The parser accepts T or the documented SQL-space separator, up to nine
fractional digits, Z, ±HH:MM, ±HHMM, and zone-less input. Output is
canonical UTC with insignificant fractional zeroes removed. To make embedded
results independent of the host timezone, Timeless defines zone-less input as
UTC, matching the pinned oracle container. VictoriaLogs otherwise consults its
process-local timezone. Addition and extreme durations use VictoriaLogs'
signed-64-bit nanosecond saturation behavior.
Missing paths, invalid timestamp strings, JSON null, numbers, booleans, arrays,
and objects remain unchanged with their native type. Exact dotted fields retain
siblings, and later pipes observe the transformed value. Adding sub-microsecond
durations to _time preserves nanoseconds in the response even though durable
log timestamps retain the configured millisecond or microsecond unit. The
operation is request-local: public rows remain byte-for-byte unchanged through
optimize, shutdown, and reopen.
Every visited row and generated string is bounded by the existing work, state,
result, response, deadline, and cancellation limits. The operation performs
one public logs scan and never accesses a shadow table. Executable
SQL-LOG-061
gives direct SQLite/libSQL users a saturating native-timestamp shift plus an
explicit sub-native nanosecond remainder. Core SQLite has no honest generic
RFC3339Nano equivalent for arbitrary metadata fields, so that semantic remains
in the Rust API rather than being mislabeled as extension support.
Exact release build db274f37b217862ef5ed35d2100675f7d1183b75
measures 64-row time_add at 3.243/3.519/3.640 ms narrow and
36.297/40.756/41.869 ms wide p50/p95/p99. Equal-cardinality controls with
the same sort, limit, projection, and public scan measure
2.884/3.197/3.303 and 32.900/35.390/36.245 ms. The 10.1%/15.2% p95 cost
is bounded RFC3339Nano parse, saturating addition, canonical formatting, and
current-row replacement. Each equal-width pair reads the identical one/four
blocks, decodes 1,024/8,192 entries, reads 235,778/1,914,055 payload bytes,
matches and returns 128/8,192 public rows, and emits 64 results. The storage
work proves that moving this language transform into an extension opcode
would not improve pruning or avoid decode/payload crossing.
generate_sequence N creates exactly N independent rows whose only field
is the decimal string _msg, from "0" through "N-1":
* | generate_sequence 3
case:=does-not-exist | generate_sequence 3 | math (_msg + 10) as value
* | generate_sequence 1_0 | offset 8
The command is case-insensitive. To match pinned VictoriaLogs v1.52.0, N
uses the shared number grammar: quoting, scientific notation, base-zero
integers, underscores, duration units, and byte units are accepted. The
parsed binary64 value must be at least one and is then truncated to an unsigned
integer, so "3.9" emits three rows. Missing, zero, sub-one, negative,
leading-plus, nonnumeric, attached, and trailing forms fail explicitly.
Generation replaces its complete input. It still runs when the source filter
matches no durable row; earlier limit, transformation, union, or query-backed
operations cannot affect its output; and the last generate_sequence wins.
Later filters, math, projection, sort, offset, limit, and statistics consume
the generated string rows normally. The operation never mutates its durable
source.
The admitted count, retained vector, decimal strings, later-pipeline work,
result rows, encoded response, deadline, and cancellation all use the existing
request limits. The planner discards every semantically dead prefix and the
reader opens no public logs cursor, so correct execution performs zero block
selection, decode, payload transfer, or public-row materialization. This is
both faster and more faithful than scanning rows merely to discard them.
Executable SQL-LOG-062 gives direct SQLite/libSQL users the complete value-generation foundation as a bounded recursive CTE. LogsQL parsing, prefix replacement, cumulative API limits, cancellation, composition, and envelopes remain Rust API concerns. There is no extension opcode: ordinary core SQL already performs the operation without reading storage, so an extension primitive could not reduce storage work.
Exact release build 8b6a5e7cb722ceeb7a5f25221994d2d8b619ed7f
measures 64 generated rows at 0.691/0.836/0.900 ms for an indexed-host source
form and 0.640/0.780/0.839 ms for a full-fixture source form p50/p95/p99.
Both responses are exactly 886 bytes. Across 50 measured requests apiece,
neither shape records a public query, native-count path, candidate block,
decoded entry, payload byte, matched entry, or returned public row. The 6.6%
lower wide-form p95 and 2.9% lower API-owned mean are loopback variation
between semantically identical scan-free execution paths. Source selectivity
cannot affect the operation.
The full evidence workload completed all 8,192 entries with zero queued work,
retained four raw blocks, 1,914,055 logical bytes, and 2,022,736 physical
database/WAL/SHM bytes. Logs RSS HWM was 107,436 KiB and metrics HWM was
50,720 KiB. QSF-245–QSF-246 retain the semantic, storage, latency, and
memory verdicts.
set_stream_fields builds a canonical VictoriaLogs stream-tag string from
the fields in each current pipeline row:
* | set_stream_fields service, host
* | set_stream_fields if (level:error) service, context*
* | fields service, host | set_stream_fields *
The command and optional if keyword are case-insensitive. A condition must
be parenthesized. At least one comma-separated field filter is required;
filters are exact names, a single trailing * prefix, or the all-field *.
Missing commas, leading/trailing commas, parenthesized field lists,
mid-name wildcards, attached command suffixes, and trailing text fail before
the public storage scan.
Selection observes the current row after all earlier pipes. Exact fields are
selected as named. Prefix and all-field filters recursively flatten retained
objects to dotted leaves; arrays stay atomic. The existing LogsQL textual
projection turns numbers, booleans, and arrays into compact text. Missing,
JSON null, empty strings, and exact object parents contribute no tag. Overlap
is deduplicated, field names are sorted bytewise, and every value is quoted
with Go strconv.Quote semantics, including \\a, \\v, \\xNN, Unicode
print categories, and lower-case Unicode escapes. The result is written to
_stream as {name="value",...}, and a matched row's _stream_id becomes
the empty string. If the optional condition is false, both current columns
are preserved unchanged.
This pipe changes only the response row. It does not declare ingestion stream
fields, compute a tenant-scoped identity, build a stream index, or mutate
durable metadata. Those separate storage semantics remain explicitly deferred
under LQL-F35, LQL-F36, and LQL-P50. Work, selected state, generated
text, result rows, response bytes, deadline, and cancellation are bounded by
the normal query limits; optimize, shutdown, and reopen preserve the source.
There is no honest portable SQL equivalent.
SQLite JSON1 can enumerate a known JSON tree, but core SQL cannot reproduce
both dynamic current-pipeline field filters and exact Go quoting. A fixed
application may concatenate known fields after applying its own compatible
quoting function, but that is application behavior rather than the public
LogsQL contract. Because this transform runs after one required public
logs scan, adding a special extension primitive would not avoid storage
reads, decode, or public-row crossing.
Exact release build bf82f7625a15170b93d2a7ea8e8fd5ec94d6300c
measures the 64-row/1,856-byte transform at 3.040/4.564/5.399 ms narrow and
38.316/40.091/45.371 ms wide p50/p95/p99. Equal-read format controls are
3.376/3.633/4.119 and 37.514/38.555/39.040 ms. Candidate p95 is 25.6%/4.0%
higher while API-owned mean is 4.4% lower/1.4% higher; every public block,
decode, payload, match, and returned-row counter is identical.
VictoriaLogs stream_context performs additional stored-stream reads around
each selected row:
* | stream_context before 2 after 3
* | stream_context after 5 time_window 30m
Timeless deliberately rejects this pipe before planning or storage because
the current public log batch and table contracts do not retain a compatible
tenant-scoped _stream_id. Grouping ordinary metadata or selecting adjacent
timestamps would mix unrelated streams and is not advertised as parity. The
HTTP API returns a source-positioned 422 response:
{
"error": "unsupported_capability",
"reason": "unsupported_logsql",
"message": "LogsQL stream_context pipe at line 1, column 5 is deferred: Timeless does not store the VictoriaLogs-compatible stream identity required for same-stream surrounding reads"
}The same explicit error applies inside nested query expressions. Quoted text,
comments, field names, and ordinary metadata values containing
stream_context remain queryable. There is no honest SQL equivalent: a
correct implementation depends on the deferred ingestion-owned stream model
and index described by LQL-F35 and LQL-F36, not a new row-level SQL recipe.
VictoriaLogs accepts leading resource options such as:
options(concurrency=2) * | stats count()
options(parallel_readers=100) _time:1d error | stats count()
These are not result-only hints. concurrency limits CPU workers inside one
query, while parallel_readers selects storage readers for that query and
each storage node. Timeless executes one public SQLite cursor sequentially on
one reader thread per query, so silently accepting either option would make a
false CPU, memory, and I/O promise. Top-level and nested uses return HTTP 422
before storage:
{
"error": "unsupported_capability",
"reason": "unsupported_logsql",
"message": "LogsQL concurrency query option at line 1, column 9 is deferred: Timeless executes each log query through one public SQLite cursor; server reader pools and request admission do not implement VictoriaLogs intra-query CPU or I/O parallelism"
}TIMELESS_LOGS_READER_CONNECTIONS controls how many independent requests may
use separate SQLite connections. Auth max_concurrent_requests limits
per-subject request admission. Those deployment controls are useful but are
not equivalent to parallel execution within one query. Direct SQLite/libSQL
applications likewise may run independent statements concurrently, but there
is no SQL statement equivalent to this resource contract.
A leading time_offset query option shifts the query's logical clock without
rewriting retained log timestamps:
options(time_offset=1h) service:=api _time:1d | sort by (_time) asc
options(time_offset=-250.5ms) level:error | fields _time, _msg
options(time_offset="1h30m", time_offset=2h) * | stats count()
options is case-insensitive; the option name time_offset is
case-sensitive. Values use the VictoriaLogs signed compound-duration grammar.
Negative, fractional, compound, and quoted durations are accepted. A leading
+, missing unit, unknown unit, missing assignment/value, malformed quoted
duration, unmatched parenthesis, or missing following query fails before any
public storage read. Repeated assignments use the last value, and a trailing
comma is accepted.
The storage interval is shifted backward by the offset and returned _time
values are shifted forward. Timeless performs source-bound arithmetic in
signed nanoseconds, then uses an exact ceiling for the inclusive lower bound
and floor for the inclusive upper bound at the table's configured millisecond
or microsecond unit. Sub-native offsets therefore never disappear through
integer truncation. Response timestamps remain canonical UTC with up to
nanosecond precision; retained rows are never mutated.
Nested query expressions inherit the surrounding offset. An inner
options(time_offset=...) replaces it rather than adding to it, including an
explicit zero. Day/week filters compose with the same logical clock. Matching
VictoriaLogs' optimizer, consecutive leading filter pipes compare original
source timestamps because they are folded into the storage predicate; a
filter after another result pipe compares the shifted _time. This order is
intentional and regression-tested.
The option uses the normal work, result-row, response-byte, deadline, and
cancellation limits. A timed-out request cancels its SQLite/Rust work and
returns the reader to the pool. generate_sequence replaces its source and
therefore has no retained timestamp to offset; aggregate-only output without
_time is unchanged after source selection.
Direct SQLite/libSQL users can apply the same exact source-bound translation
with SQL-LOG-065.
That executable recipe uses only the public logs table and carries an
explicit signed sub-native remainder for response formatting. LogsQL parsing,
nested scope, pipeline order, RFC3339Nano rendering, limits, cancellation,
and HTTP envelopes remain Rust API responsibilities; there is no
language-specific extension opcode.
Exact release build c4ea4cf6ba43de36f831e048d9f39e3e60b8d183
measures 64-row offsets at 3.230/3.538/4.159 ms narrow and
37.799/38.840/39.006 ms wide p50/p95/p99. Equal-read controls measure
3.032/3.162/3.366 and 33.530/34.322/34.499 ms. The option's p95 is
11.9%/13.2% higher, but every candidate/control pair reads exactly the same
blocks, entries, payload bytes, and public rows. The measured cost is bounded
API timestamp transformation and rendering, not storage amplification.
A leading global_filter query option applies one filter-only predicate to
every query scope created by the request:
options(global_filter=(service:=api)) level:error | sort by (_time) asc
options(global_filter=(tenant:="acme")) user:in(level:info | fields user)
options(global_filter=(region:=east)) * | join by (trace_id) (level:error)
options(global_filter=(region:=east)) * | union (level:warn)
options is case-insensitive; the option name global_filter is
case-sensitive. The value must be one parenthesized filter expression. It may
use logical groups and query-backed filters, but it cannot contain a result
pipeline. Missing assignment or value, unbalanced delimiters, an incomplete
filter, a pipeline, or trailing text fails before any public storage read.
Repeated declarations are all parsed and validated; the last valid declaration
wins, and a trailing comma is accepted.
The Rust API compiles the global value into a scoped predicate. It does not
paste query text together. The predicate is conjoined before the local filter
in the base query and in every query-backed membership, conditional pipeline,
join, and union source. A nested query inherits the surrounding predicate. An
explicit nested global_filter replaces the inherited predicate for that
scope rather than adding a second global predicate. global_filter=(*) is the
identity predicate.
Query-option timing follows declaration scope. A sibling time_offset does
not retroactively rewrite a global value in the same option list. A global
value may use its own leading options(time_offset=...), and a nested explicit
global declaration inherits the surrounding time offset present at that
declaration. These rules match the pinned VictoriaLogs initialization order.
Every independently executed public scan keeps its extension-enforced work limit, while the request also charges nested scans and retained subquery values to cumulative API work and state budgets. Deadline cancellation, result and response limits, optimize, shutdown, and reopen use the existing bounded query path; retained rows are never mutated.
Direct SQLite/libSQL users can implement the same storage behavior with
SQL-LOG-066:
repeat the shared predicate as a conjunct in every independently bounded
public logs scan. LogsQL parsing, lexical inheritance/replacement,
query-backed initialization, cumulative limits, cancellation, and HTTP
envelopes remain Rust API responsibilities. Ordinary SQL already provides the
useful storage operation, so no global-filter extension opcode is exposed.
Exact release build 72a392a0ed28f7c61c3c816a2c300c8caa588400
measures 64-row global-filter queries at 3.322/6.102/7.881 ms narrow and
35.171/38.893/39.238 ms wide p50/p95/p99. Equivalent explicit-conjunction
controls measure 3.493/5.040/7.254 and 38.350/40.365/41.518 ms. The narrow
p95 tail is 21.1% higher while its p50 is lower and its request-attributed API
mean is only 4.1% higher; the wide p95/API mean are 3.6%/6.8% lower. Both
pairs read exactly the same blocks, entries, payload bytes, and public rows
and return identical 2,560-byte responses. The differences are retained as
API/parser and whole-run variation, not storage amplification.
VictoriaLogs accepts both a leading query option and an HTTP form parameter:
options(allow_partial_response=true) service:=api | fields _time, _msg
allow_partial_response=true
This is a cluster failure-policy contract, not a relational query modifier.
With multiple vlstorage owners, VictoriaLogs may suppress an unavailable
owner when at least one other owner returns a complete response. It still
fails when every owner is unavailable or when an available owner reports a
configuration error. Timeless currently executes against one authoritative
SQLite/libSQL owner, so it has no second complete result with which to form an
honest partial response. Silently accepting the option would be a misleading
no-op; suppressing the only owner's failure would be data loss.
allow_partial_response=false is fully supported: it explicitly selects the
normal complete, fail-closed Timeless query. allow_partial_response=true
returns HTTP 422 during language planning and before storage execution:
{
"error": "unsupported_capability",
"reason": "unsupported_logsql",
"message": "LogsQL allow_partial_response is deferred: Timeless has one authoritative SQLite/libSQL storage owner; partial responses require multiple independent storage owners, unavailable-owner error classification, deterministic merge, and explicit response-completeness metadata"
}Accepted source grammar follows Go strconv.ParseBool: 1, t, T,
TRUE, true, and True; or 0, f, F, FALSE, false, and False.
Quoted values are accepted, options is case-insensitive, the option name is
case-sensitive, a trailing comma is valid, and duplicate declarations are all
validated before the final value wins. A final false value executes; a final
true value is deferred. Malformed values return HTTP 400. The same explicit
boundary survives nested membership, join, union, and global-filter parsing.
The HTTP parameter follows the same false/true rule and is never ignored; an
explicit query option overrides its valid value after both inputs are
validated, matching VictoriaLogs. An empty HTTP value is the documented
omitted/default-false form; an empty query-option value remains malformed.
False requests use the ordinary bounded query path and return complete rich
rows. Rejected true and malformed requests perform no public logs query,
count, block read, payload transfer, or decode. There is no SQL equivalent or
latency benchmark for true because ordinary SQL cannot represent multi-owner
failure suppression; a false request is simply the normal fail-closed SQL
statement. Shipping true requires multiple fenced public owners, exact
unavailable/error classes, deterministic bounded merge and ordering,
cumulative limits and cancellation, and a response contract that says which
owners contributed. No storage, extension, batching, compression, index,
transaction, retention, migration, or maintenance behavior changes for this
deferral.
The bounded stats pipeline supports textual upper-step quantile and
numeric population stddev:
* | stats quantile(0.5, duration_ms) as p50
* | stats quantile(0, duration_ms) as minimum, quantile(1, duration_ms) as maximum
* | fields duration_ms | stats quantile(0.95) as p95
* | stats stddev(duration_ms) as sigma
Function names are case-insensitive. quantile requires a decimal rank in
the inclusive range [0,1]; its fields follow the existing exact, prefix,
and all-current-field selectors, and omitted fields mean all current fields.
Selected values use the LogsQL textual projection: missing and JSON null are
empty, strings retain their contents, and other rich values use compact JSON.
Ordering follows VictoriaLogs signed integer, unsigned integer, RFC3339
timestamp, general math-number, and natural UTF-8 comparisons. For N
values, the selected zero-based rank is min(floor(phi * N), N - 1) with no
interpolation. An empty selection is the explicit empty string.
stddev uses Welford's one-pass population algorithm and divides by N, not
N - 1. In the retained Timeless typed-statistics profile, only native JSON
numbers participate. Numeric-looking strings, booleans, nulls, missing paths,
arrays, and objects are ignored. A singleton returns zero and an empty numeric
selection returns JSON null.
Every selected quantile value counts against max_work_rows; its exact text
state also counts against max_response_bytes. Every visited deviation value
counts against max_work_rows. Both operations are deadline-cancellable and
leave the SQLite reader reusable after cancellation or limit rejection.
Unlike VictoriaLogs' random reservoir above 10,000 values, Timeless never
silently makes an exact result nondeterministic: it fails with the stable
query-limit envelope.
The complete pinned VictoriaLogs fixture records four intentional retained-
model wire differences: Timeless preserves rich types, does not coerce numeric
strings for deviation, represents empty deviation as JSON null rather than a
textual NaN, and preserves an explicitly requested empty quantile field
instead of dropping it from stream JSON.
Executable
SQL-LOG-044
uses only public logs, JSON1, window functions, and a recursive Welford CTE
for one finite native-number path. Core SQLite has no honest equivalent for
the complete mixed textual natural comparator, so the full language grammar,
projection, ordering, exact state bounds, cancellation, and HTTP envelopes
remain Rust API composition. The required public scan already crosses every
selected row; no extension primitive or private storage access is involved.
Exact-build quantile evidence measures 3.279/3.636/3.834 ms narrow and
37.390/38.585/40.331 ms wide p50/p95/p99, 3.5% below/1.2% above same-run
median p95. Population deviation measures 3.330/3.517/3.676 and
36.149/37.372/37.815 ms, 2.9%/0.1% below same-run average p95. Every
equal-width pair reads the same one/four blocks, decodes the same
1,024/8,192 entries, transfers the same extension payload bytes, and
materializes the same 128/8,192 public rows. QSF-193 therefore retains the
bounded API implementation and the complete measured tails without adding a
storage primitive.
The bounded stats pipeline supports sum_len(fields...):
* | stats sum_len(_msg) as message_bytes
* | stats sum_len(context*) as context_bytes
* | fields _msg, service | stats sum_len() as selected_bytes
Function names are case-insensitive. Fields may be exact names or suffix-
prefix selectors such as context*; empty parentheses select every field in
the current projected row. Each selected value contributes the byte length of
its textual projection: missing and JSON null contribute zero, strings use
their raw UTF-8 bytes, and numbers, booleans, arrays, and objects use compact
JSON text. Thus "é" contributes two, not one. Timeless returns a native JSON
integer; the pinned VictoriaLogs endpoint returns the same unsigned value as
decimal text.
Every selected traversal counts against max_work_rows, including a missing
or null selection that contributes zero. The aggregate keeps one checked
u64; overflow, limits, deadlines, and cancellation fail explicitly and
leave the SQLite reader reusable. Stored log rows remain immutable across
flush, optimize, shutdown, and reopen.
Executable
SQL-LOG-045
uses only public logs, JSON1, length(CAST(... AS BLOB)), and SUM for one
exact metadata path. Dynamic field expansion, canonical _time, _msg, and
level, grammar, checked unsigned overflow, limits, cancellation, and HTTP
envelopes remain bounded Rust API composition. The required public scan
already crosses each selected row, so no extension primitive or private
storage access is involved.
Exact-build evidence measures 3.223/3.460/3.940 ms narrow and
34.152/35.691/36.932 ms wide p50/p95/p99. Same-run numeric-sum controls
measure 3.109/3.719/4.321 and 35.066/37.760/40.220 ms, making sum_len p95
7.0%/5.5% lower. Every equal-width pair reads the same one/four blocks,
decodes the same 1,024/8,192 entries, transfers the same
235,778/1,914,055 extension payload bytes, and materializes the same
128/8,192 public rows. QSF-195 retains this bounded API reduction without a
storage primitive.
The bounded stats pipeline supports one-value selection and extrema that
return a field from the selected row:
* | stats any(service) as one_service
* | stats field_min(duration_ms, payload) as fastest_payload
* | stats field_max(duration_ms, payload) as slowest_payload
Function names are case-insensitive. any requires exactly one exact field,
skips missing, JSON null, and empty strings, and returns the first qualifying
value in current-pipeline order. Zero, false, arrays, and objects qualify and
retain their native JSON type. VictoriaLogs promises an arbitrary value and
its answer may change with physical encoding; Timeless deliberately provides
the stronger deterministic result.
field_min(source,result) and field_max(source,result) require two exact
fields. Rows with an empty source are skipped. The source uses VictoriaLogs'
signed integer, unsigned integer, RFC3339 timestamp, general math-number, and
natural UTF-8 comparison chain. Equal source values keep the first current
row. The companion result retains its native JSON type. A missing companion is
the explicit empty string, while JSON null, an empty string, arrays, and
objects remain distinct. An empty candidate set produces an empty string.
Every visited candidate counts against max_work_rows; retained comparison
keys and rich companion values count against max_response_bytes, including
their bounded nested traversal. Deadlines and cancellation fail explicitly
and leave the SQLite reader reusable. The functions do not mutate durable
source rows and preserve their results across flush, optimize, shutdown, and
reopen.
Executable
SQL-LOG-046
uses only public logs, JSON1, explicit ordering, and window functions for a
deterministic exact-path any and finite-native-number extrema. Core SQLite
does not implement the complete LogsQL textual comparator, canonical fields,
or the API's retained rich policy, so those remain bounded Rust composition.
The required public scan already materializes every candidate row; no private
table or language-specific extension primitive is involved.
Exact release build 9a0303ff2f9820fb5a20da3686c30fdb33595d7c
measures any at 3.077/3.293/3.597 ms narrow and 32.767/33.764/36.128 ms
wide p50/p95/p99. Equal-output min controls measure 3.085/4.476/8.065 and
34.858/37.518/38.095 ms, making any p95 26.4%/10.0% lower. Companion
field_min plus field_max measure 3.182/3.306/3.849 and
34.004/36.738/42.084 ms versus 3.356/3.704/4.390 and
37.000/38.270/38.988 ms for equal-output numeric extrema controls, or
10.7%/4.0% lower p95. All pairs read the same one/four blocks, decode the same
1,024/8,192 entries, transfer the same 235,778/1,914,055 extension bytes, and
materialize the same 128/8,192 public rows. The 42.084 ms wide extrema p99 is
retained honestly. The checked evidence artifact is
2026-08-06_session17_lql_s10_any_field_extrema.json
with SHA-256
711d80e590550a2e4655103a60b7ec3da0c9fda84fb2f9dcdd1500428c674d2f.
The bounded stats pipeline can return a complete selected field set from
one qualifying row:
* | stats row_any(service, payload*) as representative
* | stats row_min(duration_ms) as fastest_row
* | stats row_max(duration_ms, service, payload) slowest_row
Function names are case-insensitive. row_any(fields...) accepts exact,
flattened-prefix, or all-field selectors; empty parentheses select all current
fields. It returns the first current row where at least one selected field is
nonempty, projected to the complete selected object. Prefixes recursively
match flattened leaf paths and reconstruct nesting. Missing selected paths are
omitted, while selected JSON null, empty strings, false, zero, arrays, empty
objects, and other objects keep their native types. If no row qualifies, the
result is {}. VictoriaLogs permits merge-order-dependent selection; Timeless
documents deterministic current-pipeline order instead.
row_min(source[, fields...]) and row_max require one exact comparison
field. With no result selectors they return all current fields. Otherwise the
result selectors may be exact, flattened-prefix, or all fields. Empty source
values are skipped; comparison follows VictoriaLogs' signed integer, unsigned
integer, RFC3339 timestamp, general math-number, and natural UTF-8 chain.
Equal comparison values retain the first current row. Empty input returns
{}. An alias may be written as either as result or the upstream shorthand
result; additional trailing tokens fail explicitly.
Every candidate, selected traversal, flattened-prefix node, retained comparison key, and cloned rich result counts against the query work/state limits. Deadline cancellation, limit errors, and malformed syntax leave the public SQLite reader reusable. Source rows remain immutable across flush, optimize, shutdown, and reopen.
Executable
SQL-LOG-047
uses only public logs, JSON1, fixed exact paths, and explicit row ordering
for a deterministic rich-row selection and finite-native-number row extrema.
Dynamic selectors and the complete LogsQL comparator remain bounded Rust API
composition after the same required public scan; no private table or language-
specific extension primitive is involved.
Exact release build 74a92f1b6ae927695fbb39d80303482966218e10
measures row_any at 2.880/3.097/4.720 ms narrow and
35.036/37.829/39.595 ms wide p50/p95/p99. Same-scan scalar any controls
measure 3.201/3.660/4.240 and 36.597/37.888/39.081 ms, making row_any p95
15.4%/0.2% lower. Two rich row extrema measure 2.948/3.219/3.427 and
37.970/39.790/40.474 ms versus 3.022/3.429/3.737 and
34.149/36.227/37.943 ms for scalar companion-extrema controls, or 6.1% lower/
9.8% higher p95. Rich outputs are intentionally larger: 36 versus 12 bytes
for selection and 101 versus 26 bytes for extrema. Every comparison reads the
same one/four blocks, decodes the same 1,024/8,192 entries, transfers the same
235,778/1,914,055 extension bytes, and materializes the same 128/8,192 public
rows. The checked evidence artifact is
2026-08-06_session17_lql_s11_row_selection.json
with SHA-256
eae8dc26fc6eb257445b96229a74891fea695470c65d053724dc53dd0184ed8b.
The statistics pipeline can return selected current-row objects as one JSON array string:
* | stats json_values(host, status, context*) sort by (status desc, host) limit 100 as rows
* | json_values(range_key, context.attempt) order (range_key desc) limit 64 values
* | stats json_values() as all_rows
json_values is case-insensitive and works both as a statistics expression
and as the standalone pipeline shorthand. Arguments may be exact quoted or
dotted fields, suffix-wildcard prefixes, or *; empty parentheses select all
current fields. Repeated selectors are an idempotent union. Each selected row
becomes one object. Missing exact paths are omitted, selected explicit nulls,
empty strings, false, zero, arrays, empty objects, and nested objects retain
their native JSON type and shape. A row with no selected field contributes
{}, and empty input still emits one statistics row containing the string
[].
Optional sort or order, optional by, and a parenthesized comma-separated
list of exact fields select the ordering. Each field defaults to ascending and
may specify asc or desc. Comparison follows the VictoriaLogs signed,
unsigned, RFC3339, duration, byte-size, math-number, and natural UTF-8 order;
missing and null sort as empty text. Timeless strengthens otherwise
unspecified equal-key order to deterministic current public-row order. Without
a sort clause, current public-row order is retained; no compatibility claim is
made for VictoriaLogs' physical cross-block merge order.
A positive limit selects a bounded top-k and must not exceed the server's
max_result_rows. limit 0 and an omitted limit mean no operator-specific
limit, but the hard result cap still applies and rejects an oversized source
instead of silently truncating it. Sort-key projection, nested selection,
retained heap/indices, encoded strings, multiple json_values expressions,
result rows, response bytes, deadline, and cancellation are cumulatively
bounded. A cancelled or rejected request leaves the public SQLite reader
reusable, and request-local selection never mutates durable rows.
An explicit alias may use as name or the upstream bare-name shorthand. With
no alias, the result field is the normalized function spelling: order
becomes sort by, explicit asc is omitted, quoted/underscored limits become
decimal, and zero is omitted. For example, json_values(case) order (a asc) limit "2" produces the field name json_values(case) sort by (a) limit 2.
The result field is deliberately a JSON string containing an array, matching
the pinned VictoriaLogs wire contract. VictoriaLogs stores and emits textual
columns; Timeless preserves the richer types already retained by its log
model. Executable
SQL-LOG-063
provides direct SQLite/libSQL users the fixed-path, native-number-sort JSON1
foundation. Dynamic selectors, the complete natural comparator, bounded
top-k, LogsQL grammar, limits, cancellation, and envelopes remain Rust API
composition over public logs rows. No private table, extension opcode,
storage-format change, or batching change is involved.
Exact release build 898006684a82b5fd6cc0f7ff477c75e5c1778367
measures the 64-object natural top-k at 3.275/3.557/3.967 ms narrow and
35.728/37.237/37.801 ms wide p50/p95/p99. Equal-scan bounded values
controls measure 3.199/3.364/3.429 and 33.646/34.563/37.043 ms. Candidate
p95 is 5.7%/7.7% higher and request-attributed API means are 4.5%/6.4%
higher. Candidate responses are 3,663 bytes rather than 451 because they
retain 64 typed nested objects. Every equal-width pair reads byte-identical
public storage work: one/four blocks, 1,024/8,192 decoded entries,
235,778/1,914,055 payload bytes, and 128/8,192 materialized rows. The checked
artifact and accepted boundary verdict are recorded in QSF-248.
The statistics pipeline can aggregate one exact current-row field into the VictoriaMetrics logarithmic bucket layout:
* | stats histogram(duration_ms) as duration_buckets
* | histogram(context.attempt)
* | stats HiStOgRaM("literal dotted field") as buckets
histogram is case-insensitive and works as a statistics expression or
standalone shorthand. It requires exactly one exact field; empty arguments,
*, suffix-wildcard prefixes, multiple fields, limit, attached command
suffixes, and trailing tokens fail explicitly. Without an alias the result
field is the canonical lowercase spelling, such as
histogram(context.attempt). Quoted names select a literal field name;
unquoted dotted names traverse retained nested metadata.
Native JSON integers/reals and VictoriaLogs-compatible textual decimals,
general math numbers, compound durations, and byte sizes are numeric input.
IPv4 and RFC3339 timestamp strings are deliberately not treated as numbers.
Negative values, NaN, missing, null, booleans, arrays, and objects are
ignored. +Inf enters the upper bucket and -Inf is ignored. Rich source
values and nested siblings remain unchanged.
The layout contains 486 middle buckets—18 per decimal decade from 1e-9
through 1e18—plus 0...1.000e-09 and 1.000e+18...+Inf. Intervals are
lower-exclusive and upper-inclusive. Exact 1e-9 enters the first middle
bucket; exact later boundaries enter the preceding bucket. Only nonempty
buckets are emitted, naturally ordered by vmrange. The field value is one
string containing compact JSON with native integer hit counts:
[{"vmrange":"1.896e+00...2.154e+00","hits":2}]Empty input or input with no accepted value returns the string []. The
fixed 488-counter state, every visited row, multiple histogram expressions,
the ordered bucket vector, encoded result, response, deadline, and
cancellation are cumulatively bounded. A rejected or cancelled query leaves
the public reader reusable; flush, optimize, shutdown, and reopen do not
change results.
Pinned VictoriaLogs v1.52.0 source and all 1,388 oracle cases establish the
grammar, input coercions, exact boundaries, natural order, result envelope,
and explicit errors. Executable
SQL-LOG-064
provides a fixed-path native-number foundation using public logs, SQLite
math, grouping, and JSON functions. Textual duration/byte parsing, natural
result ordering, dynamic LogsQL syntax, limits, cancellation, and HTTP
envelopes stay in the Rust API. The operation already requires the public row
scan, so no private table or extension primitive is justified.
Exact release build 63081644c87fa67fe1c9874ea375195146262433
measures histogram aggregation at 3.317/4.329/5.812 ms narrow and
37.928/39.536/42.954 ms wide p50/p95/p99. Equal-scan bounded values
controls measure 3.341/3.517/3.611 and 37.326/38.826/44.811 ms. Histogram
p95 is 23.1%/1.8% higher and request-attributed API means are 6.9%/2.0%
higher. The 268/278-byte candidate responses contain five nonempty bucket
objects; the controls are 164 bytes.
Every pair reads byte-identical public storage work: one/four blocks,
1,024/8,192 decoded entries, 235,778/1,914,055 payload bytes, and
128/8,192 materialized rows. The checked artifact and accepted bounded-
composition verdict are recorded in QSF-250.
Embedded hosts can inspect log storage and schedule maintenance through the public statistics TVF without depending on private block, term, or metadata tables:
SELECT key, value
FROM timeless_stats('logs')
WHERE key IN (
'timestamp_unit',
'blocks', 'raw_blocks', 'compressed_blocks',
'buffered_entries', 'disk_entries', 'total_entries',
'bytes_on_disk', 'raw_bytes', 'compressed_bytes',
'terms', 'index_bytes',
'ts_min', 'ts_max',
'optimize_source_entries', 'optimize_source_bytes'
)
ORDER BY key;Timestamps use the table's declared timestamp_unit. total_entries includes
the live buffer, while disk_entries does not. The three payload-byte rows
measure stored block blobs rather than the complete SQLite database, WAL, or
freelist. terms is a posting-row count. index_bytes is the SQLite page
allocation for the log term/timestamp/metadata structures and is NULL when
the SQLite build does not expose dbstat. The optimize_source_* rows describe
the raw or undersized persisted blocks currently eligible as optimizer source;
they are an observation for choosing a bounded optimize:<entries> command,
not a separate storage contract. Statistics keys are additive, so callers
should select the keys they understand and tolerate new rows.
The extension alone owns and interprets its shadow tables. A signal server or embedded application that needs a missing statistic should extend this public surface instead of reading private tables or duplicating block policy.
The Rust metrics API accepts Prometheus selectors that identify series only
by labels. For example, this instant query selects every metric name whose
series has job="api":
GET /prometheus/api/v1/query?query=%7Bjob%3D%22api%22%7D&time=1700100010
The planner first reads metric names and matching series through
timeless_series, without decoding chunks. It then issues one bounded,
exact-metric timeless_raw_frame read for each selected name and composes the
Prometheus result in Rust. Metric names and canonical labels determine stable
output order. Label matchers retain anchored-regex and missing-as-empty
semantics; the upstream-invalid {job=~".*"} form fails because every matcher
can match the empty string.
Direct SQL users perform the same two public steps: enumerate candidate
name values with timeless_series('metrics'), then bind each name to
SQL-PROM-001.
There is intentionally no PromQL parser or special nameless-selector opcode in
the extension.
__name__ uses the same anchored matcher rules as labels. Regex and negative
forms remain API planning rather than extension syntax:
{__name__=~"http_.+",job="api"}
{__name__!="http_debug",job="api"}
{__name__=~"http_.+",__name__!~"http_internal_.+",job="api"}
Repeated __name__ matchers are ANDed. A name matcher that can match the empty
string does not by itself make a nameless selector legal, so
{__name__!="missing"} fails instead of silently selecting nearly everything.
Catalog rows are tested against every name matcher before any metric payload
is requested.
Prometheus 3 quoted metric and label names work through the public text-ingest and query paths:
{"http.request/duration-秒","node.name"="東京"}
{"oracle.\"quoted\"\\温度","node.name"="大阪"}
The exposition parser preserves decoded UTF-8, quote, and backslash bytes as
series identity. The same identity survives timeless_series discovery,
compact, shutdown, and reopen. Direct SQLite/libSQL users bind the decoded
metric name as ordinary TEXT and the decoded label key in the matcher JSON;
SQL-PROM-001
is the executable equivalent.
Line comments begin with # outside a quoted string and continue to the next
newline. They may occur before, after, or between expression tokens:
# compare the current request rate
sum by (service) (
rate( # one bounded counter window
http_requests_total[5m]
)
) # trailing explanation
Comments are API syntax and require no extension primitive. Error and warning
positions still refer to the original query text; parentheses, calls, and
brackets inside a comment do not participate in source scanning, while #
inside a quoted matcher remains part of its value. A comment-only query fails
as bad_data rather than becoming an empty expression.
Temporal modifiers change where a selector reads without changing the outer evaluation timestamps returned by an instant-vector or range query:
http_requests_total offset 5m
http_requests_total offset -30s
http_requests_total @ 1700300010
http_requests_total @ start()
http_requests_total @ end() offset 10s
The planner resolves @ first and subtracts the signed offset second. Numeric
@ supports millisecond precision; start() and end() use the request's
outer range endpoints. Lookback and range windows are relative to that resolved
lookup time. Root range vectors keep stored sample timestamps, while selector
and function results keep the outer evaluation grid. See executable direct-SQL
forms in
SQL-PROM-008.
The Rust metrics API evaluates shipped instant-vector expressions over a globally aligned inner grid:
http_requests_total[30m:30s]
avg_over_time(http_requests_total[30m:30s])
min_over_time(http_requests_total[30m:30s])
max_over_time(http_requests_total[30m:30s])
sum_over_time(http_requests_total[30m:30s])
count_over_time(http_requests_total[30m:30s])
last_over_time(http_requests_total[30m:30s])
avg_over_time(http_requests_total[30m:])
avg_over_time(http_requests_total[30m:30s] @ end() offset 5m)
avg_over_time(avg_over_time(http_requests_total[5m:30s])[30m:1m])
The interval is (effective_time-range,effective_time]; explicit resolutions
are aligned to Unix epoch multiples rather than the outer query start. When
the resolution is omitted, the API uses
TIMELESS_METRICS_PROMQL_DEFAULT_SUBQUERY_STEP_MS (15 seconds by default).
Subquery @ is resolved against the original request start/end before signed
offset is subtracted. Root subqueries are range vectors and therefore work
only on an instant-query endpoint; range functions return their result on the
outer instant/range grid.
Intermediate points share the hard max_work_points bound, serialized inner
matrices share the response-byte bound, and cancellation is checked during
inner execution, decode, and folding. Direct SQLite/libSQL callers can build a
selector subquery with the executable, pre-epoch-safe alignment recipe in
SQL-PROM-009.
PromQL syntax and arbitrary AST composition remain in Rust, not the extension.
Unary minus accepts scalar or instant-vector expressions, including nested shipped expressions:
-1
-http_requests_total{job="api"}
-avg_over_time(http_request_duration_seconds_sum[5m])
-(-http_requests_total)
Vector samples retain every non-name label and their evaluation timestamps,
but Prometheus removes __name__ even for a double negation. Range evaluation
returns the usual matrix, and scalar/vector NaN, +Inf, and -Inf values use
the Prometheus string representation. Each composed child point counts toward
the configured intermediate-work limit; cancellation is checked while
decoding, transforming, and serializing the bounded result.
Direct SQLite/libSQL users can apply the equivalent ordinary numeric operation
with
SQL-PROM-010. The SQL
recipe is the stored-vector foundation, not a claim that SQLite result typing
or IEEE rendering is a PromQL envelope.
The Rust API evaluates all stable float arithmetic combinations:
2 ^ 8
http_requests_total * 1000
100 - queue_depth
errors_total / requests_total
Scalar/scalar returns a scalar. Either vector/scalar direction returns the
vector's labels and grid; vector/vector uses one-to-one matching on all labels
except __name__, emits only timestamps present on both sides, and uses the
left labels. Every vector arithmetic result removes the metric name. Duplicate
match signatures at the same evaluation timestamp fail as an execution error
instead of producing a Cartesian product. IEEE division, modulo, and power
results use Prometheus NaN/+Inf/-Inf strings.
Both operands execute once across the requested grid. Their cumulative child
points count as bounded intermediate work, and cancellation is checked during
each child read, matching step, arithmetic operation, and serialization. The
public SQL foundations—including vector/scalar arithmetic and an exact-label
join—are executable in
SQL-PROM-004.
The Rust layer remains responsible for AST precedence, cardinality errors,
result types, labels, limits, cancellation, and Prometheus envelopes.
PromQL atan2 uses the same matching and result rules, with the left operand
as Y and the right operand as X:
vertical_displacement atan2 horizontal_displacement
queue_depth atan2 2
It supports scalar/scalar and either scalar/vector direction, default and
explicit vector matching, and range grids. Vector results remove the metric
name. The Rust evaluator uses deterministic Go-compatible arithmetic because
SQLite/C and Go can disagree by one last-place bit for otherwise identical
inputs. Direct SQLite/libSQL users can use ordinary atan2(Y,X) through the
bounded scalar/vector and label-join recipes in
SQL-PROM-055; the recipe
documents that last-bit boundary honestly.
All six float comparisons work across scalar/scalar, either scalar/vector direction, and matched vectors:
queue_depth > 100
0 <= temperature_celsius
errors_total != requests_total
latency_seconds > bool 0.5
Scalar/scalar comparisons require bool and return a scalar 0 or 1.
Without bool, a vector comparison is a filter: false samples disappear and
true samples retain the vector's original value and metric name. With bool,
every matched sample becomes 0 or 1 and __name__ is removed. Vector
matching, sparse timestamps, duplicate errors, cumulative work, and
cancellation use the same contracts as arithmetic.
Direct SQL users can express both forms with a public-grid WHERE predicate
or a CASE/boolean cast; see executable
SQL-PROM-011.
and, or, and unless compare instant vectors by every non-name label at
each evaluation timestamp. They are true many-to-many membership operations,
not arithmetic joins:
request_errors and requests_total
request_errors unless maintenance_targets
primary_measurements or fallback_measurements
and retains every matching left sample. unless retains every unmatched
left sample. or retains every left sample and adds only right samples whose
matching signature is absent on the left at that step. The contributing
sample keeps its value, labels, and metric name. A range query repeats this
decision independently on every grid point, so a right series can appear only
at steps where no matching left sample exists.
Both child vectors execute once. Their points count toward the cumulative
intermediate-work limit, and membership/output loops check cancellation. Set
operators reject scalar and matrix operands explicitly. Direct SQLite/libSQL
users can implement the exact membership foundation with EXISTS,
NOT EXISTS, and a left-preferred UNION ALL; see executable
SQL-PROM-012.
Vector/vector arithmetic, comparisons, and set operators accept both stable matching modifiers:
errors_total / on(service, method) requests_total
desired_replicas == ignoring(source) current_replicas
primary or on(instance) fallback
on(...) builds the match key from exactly the listed labels; a missing label
has the empty-string value. on() therefore places every present sample at a
step in one match group. ignoring(...) builds the key from every label except
__name__ and the listed labels; ignoring() is default matching. Duplicate
keys fail one-to-one arithmetic/comparisons, while set operators deliberately
remain many-to-many.
For a one-to-one result, on(...) retains only the named labels and
ignoring(...) removes the ignored labels from the left result. Arithmetic
and bool also remove __name__; an on(...) comparison filter has only its
named labels. Set operators do not project labels and retain the complete
contributing side. Matching is repeated per evaluation timestamp and is
covered by the same cumulative work, result, byte, deadline, and cancellation
limits as default matching.
Direct SQLite/libSQL users can make the label key explicit with SQLite JSON
functions over public grids; executable
SQL-PROM-013
shows both forms.
group_left and group_right explicitly allow multiple series on one side of
an arithmetic or comparison match:
pod_errors + on(service) group_left(team) service_budget
service_budget - on(service) group_right(team) pod_usage
group_left makes the right side unique and retains each left-side series;
group_right makes the left side unique and retains each right-side series.
The optional label list copies labels from that unique “one” side. A missing
or empty included label removes the corresponding many-side label. The copy
must still leave every result labelset unique at each evaluation timestamp.
Operation direction never changes: a - ... group_right ... b still computes
a - b. For a non-bool comparison, the surviving sample value is likewise
the original left value; group_right uses the right metric identity solely
to represent its right-side result cardinality. Arithmetic and bool follow
their established metric-name removal rules.
The evaluator validates the one side and result uniqueness independently at
every step, supports an active one-side series changing across a range, and
keeps all child/result work bounded and cancellable. Direct SQL users can
compose the public grids and run an explicit uniqueness preflight; executable
SQL-PROM-014
shows the complete foundation.
The Rust metrics API aggregates every shipped instant-vector expression at each evaluation timestamp:
sum(http_requests_total)
sum by (service) (http_requests_total)
sum without (instance, pod) (http_requests_total)
avg by (service) (request_duration_seconds)
min by (service) (queue_depth)
max without (instance) (queue_depth)
Without a modifier, all samples form one empty-label group. by keeps only
the named, non-empty labels; a missing grouping label therefore joins the
empty group. without removes its labels and always removes __name__.
Naming __name__ in by explicitly retains it, matching Prometheus. Range
queries repeat grouping per outer grid timestamp, so sparse inputs do not
invent samples. NaN propagates, like-signed infinities remain infinite, and
opposite infinities produce NaN.
avg uses compensated summation for cancellation-prone finite values and
switches to a compensated incremental mean before a running-sum overflow.
This preserves a finite mean for inputs such as two f64::MAX samples while
retaining Prometheus NaN and infinity behavior.
min and max ignore NaN when another value exists, retain signed
infinities, and return NaN for an all-NaN group.
count includes every selected sample regardless of its numeric value;
group returns one for every non-empty group. Both accept the same
by/without modifiers and produce no output for empty input.
stdvar is the population variance and stddev its square root. The API uses
Prometheus's Welford update, returns zero for a singleton, and returns NaN
when any group member is NaN or infinite.
topk and bottomk evaluate their scalar parameter and rank independently at
every timestamp and grouping partition while retaining each selected series'
complete original labels and metric name. Fractional k truncates toward
zero; values below one return no series; NaN and positive overflow are errors.
Numeric values outrank NaN in both directions, so NaN is returned only when a
partition has fewer than k numeric samples.
Prometheus does not define which equal-valued series wins a cutoff tie;
Timeless uses canonical labels as a deterministic tie-break without claiming
that label choice as cross-engine language behavior.
quantile(q, vector) sorts each group at each evaluation step and linearly
interpolates rank q * (N - 1). Raw NaN participates at the low end of the
rank; q < 0, q > 1, and NaN q produce -Inf, +Inf, and NaN.
count_values("label", vector) emits one count per distinct sample value and
group at each step. The selected label overwrites an input label of the same
name before grouping. Values use Prometheus fixed shortest formatting,
including -0, infinities, NaN, and fully expanded decimal exponents; this
operation can approach input cardinality and remains subject to result limits.
Prometheus 3.13.2 classifies limitk and limit_ratio as experimental
aggregators. The stable Timeless PromQL endpoint rejects both with the pinned
promql-experimental-functions feature-gate diagnostic on GET and POST,
including after reopen, and performs no storage query. There is no hidden SQL,
extension, Elixir, or process fallback. Portable SQLite/libSQL cannot reproduce
Prometheus's canonical label hashing and evaluator-order selection honestly;
direct users may write an explicitly application-defined ORDER BY ... LIMIT
query, but it is not a PromQL equivalent. A future implementation requires a
separately enabled experimental Rust API tier and matching oracle gate.
Binary fill, fill_left, and fill_right modifiers are independently
feature-gated by Prometheus 3.13.2. Stable Timeless GET and POST requests now
return its pinned binop fill modifiers are experimental and not enabled
diagnostic before storage, including after reopen. There is no implicit
MetricsQL or fallback execution. Direct SQLite/libSQL users can perform
bounded one-to-one float filling with executable
SQL-PROM-057,
which uses two public grids and an ordinary full-outer composition. It does not
claim the experimental PromQL grammar, group matching, labels, types, limits,
or envelopes.
Prometheus's stable </ and >/ operators trim observations inside a typed
native-histogram sample; they are not comparison or division spellings. The
current Timeless metrics model stores float samples only, with classic
histograms represented as separate _bucket float series. Stable GET and
POST requests containing either trim operator therefore fail at the operator
position with requires typed native-histogram storage, before any storage
query and with the same result after reopen. Operators inside quoted strings
or line comments do not trigger that rejection. There is no SQL equivalent:
classic bucket rows do not encode one native sample's exponential/custom
schema, positive and negative spans, zero bucket, count, sum, or reset hint.
PQL-O19 remains deferred with PQL-S22 as its prerequisite.
mad_over_time(range-vector) is experimental in pinned Prometheus 3.13.2.
Stable Timeless GET and POST requests return the pinned function "mad_over_time" is not enabled diagnostic before storage, including after
reopen; there is no hidden experimental or Elixir fallback. Direct
SQLite/libSQL users can compute the finite-float statistic with executable
SQL-PROM-058, which
applies linear median ranking to a bounded public raw window and again to its
absolute deviations. That SQL foundation is not a stable PromQL support claim
and documents the remaining NaN, signed-zero, native-histogram, annotations,
labels, limits, cancellation, and envelope work.
Pinned Prometheus likewise feature-gates ts_of_first_over_time,
ts_of_last_over_time, ts_of_min_over_time, and ts_of_max_over_time.
Stable Timeless preserves each disabled-function diagnostic before storage on
GET, POST, and reopen. Direct SQLite/libSQL users can bind first, last,
min, or max in executable
SQL-PROM-059
to obtain the finite-float source timestamp over bounded public raw windows;
min/max ties choose the latest sample. The SQL recipe states timestamp units
and the native-histogram/NaN limitations explicitly and is not relabeled as
stable PromQL support.
sort_by_label(vector, "label", ...) and sort_by_label_desc are
experimental in pinned Prometheus 3.13.2. Stable Timeless preserves both
disabled-function diagnostics before storage on GET, POST, and reopen. No SQL
equivalent is claimed: SQLite's portable text order is lexicographic, while
Prometheus uses natural order for every requested label and its exact complete
label-set comparator for ties. A future experimental Rust tier can sort the
already-bounded child vector; there is no extension, Elixir, or process
fallback.
info(vector[, {label_matchers...}]) is likewise experimental. Stable
Timeless returns Prometheus's disabled-function error before evaluating the
child or reading storage. No full SQL equivalent is advertised: upstream
performs an additional lookback-aware info-series read, resolves data-label
churn by source timestamp, observes exact stale markers, and supports
native-histogram base samples. Application code may perform a simpler metadata
join through public catalog/raw rows, but it must not call that join PromQL
info(). A future Rust experimental tier requires the PQL-S17 stale-marker
and PQL-S22 typed-histogram prerequisites; there is no Elixir, NIF, HTTP, or
process fallback.
The API evaluates the bounded child once, checks cancellation while grouping,
and charges every child point to the cumulative intermediate-work limit.
Storage remains unchanged. Direct SQLite/libSQL users use ordinary SUM over
the public grid in executable
SQL-PROM-003.
The corresponding ordinary-SQL average is executable
SQL-PROM-015;
the API adds Prometheus's compensated edge arithmetic.
Direct SQL extrema are in executable
SQL-PROM-016,
including the ordinary-SQL versus packed-NaN distinction.
Cross-series count and presence use executable
SQL-PROM-017,
which deliberately uses COUNT(*) so IEEE NaN rows are not lost as SQL NULL.
For finite, well-scaled data,
SQL-PROM-018
provides an executable second-moment recipe and states why the API's Welford
arithmetic is required for exact edge semantics.
Step-local ranking is executable as ordinary window-function SQL in
SQL-PROM-005.
Finite cross-series interpolation is executable in
SQL-PROM-019;
the recipe states why packed raw bits are needed for PromQL's NaN rank.
Raw numeric grouping for direct users is executable in
SQL-PROM-020,
with Prometheus label formatting correctly retained in the API.
SELECT series_id, labels, value
FROM timeless_aggregate(
'metrics', 'cpu_usage', '{"env":"prod"}', :t0, :t1, 'avg');
-- operations: avg | sum | min | max | countBounds are inclusive. Each non-empty matched series produces one row; empty
series and empty ranges produce none. count is a SQLite INTEGER. Fully
covered chunks use their persisted count/sum/min/max metadata and only partial
boundary chunks are decoded. As a result, sum and avg use chunk-local
left-to-right accumulation followed by chunk-index order; a completely flat
SQL scan can differ by normal floating-point rounding.
NaN handling is explicit: every NaN is included in count; any NaN propagates
through sum and avg and surfaces as SQL NULL; min and max ignore NaNs
when a numeric value exists and otherwise return NULL. Label matchers have
the same equality/anchored-regex/negative semantics as the other metric TVFs.
SELECT series_id, labels, ts, value
FROM timeless_latest(
'metrics', 'cpu_usage', '{"env":"prod"}', :t0, :t1);Bounds are inclusive and every non-empty matched series emits at most one row. The greatest timestamp wins. If several points share that timestamp, the first point in stable raw engine order wins: chunk-index order, then in-chunk order, then buffered insertion order. The engine searches candidate chunks by newest possible timestamp and stops when an older chunk cannot change the winner.
New chunks persist the first value at their maximum timestamp as nullable
metadata, avoiding decompression for the common unbounded-latest query. On
reopen, databases created by an older extension add the column automatically;
old rows keep NULL and use the exact decode fallback until compaction.
- Use ordinary row TVFs for SQL joins, filtering, ordering, and modest result sets.
- Use the
*_batchesTVFs when a host wants one independently consumable blob per series, especially for raw, window, or rollup streams. - Use a whole-result
*_frameTVF when a high-cardinality host or remote boundary wants to fetch one columnar aggregate, latest, or raw result with a single SQLite row.
Detect additive modules through SQLite rather than comparing extension version strings:
SELECT name
FROM pragma_module_list
WHERE name IN ('timeless_aggregate_frame', 'timeless_latest_frame')
ORDER BY name;Both rows mean both frame APIs are available. A host can prepare and reuse the same ID-selected statement in the normal SQLite fashion; no extension-specific binding API is involved:
read_one = db.cursor()
sql = """SELECT value FROM timeless_aggregate(
'metrics', 'cpu_usage', NULL, ?, ?, 'avg')
WHERE series_id = ?"""
value = read_one.execute(sql, (start, stop, series_id)).fetchone()Resolve a catalog ID once, cache it in the host, and use ordinary equality constraints on later reads:
SELECT series_id
FROM timeless_series('metrics', 'cpu_usage', '{"host":"web-1"}');
SELECT ts, value
FROM timeless_raw('metrics', 'cpu_usage', NULL, :t0, :t1)
WHERE series_id = :series_id;
SELECT value
FROM timeless_aggregate('metrics', 'cpu_usage', NULL, :t0, :t1, 'avg')
WHERE series_id = :series_id;
SELECT ts, value
FROM metrics
WHERE series_id = :series_id AND ts BETWEEN :t0 AND :t1;series_id = ? pushes into the base metrics table, timeless_series, and
every per-series metrics TVF. On the row-oriented grid, window, and rollup
TVFs it is an explicitly selectable hidden column, so old SELECT * shapes and
function arities do not change. The constraint intersects with the TVF's table,
metric, and matcher arguments: an ID from another metric or one rejected by the
filter returns no rows and reads no chunks.
The same constraint composes with catalog-driven joins:
SELECT s.labels, q.ts, q.value
FROM timeless_series('metrics', 'cpu_usage', '{"env":"prod"}') AS s
JOIN timeless_latest('metrics', 'cpu_usage', NULL, :t0, :t1) AS q
ON q.series_id = s.series_id;IDs are durable and table-scoped. They survive flush, reopen, backup, restore,
compaction, and retention, but an ID from one independently created database
must not be used in another. INTEGER affinity is honored (1, 1.0, and
'1' select the same handle); NULL, non-integral, malformed, and missing IDs
match nothing. The initial API intentionally supports equality only, not
IN (...).
timeless_aggregate_frame has the same arguments and semantics as
timeless_aggregate, but emits one row for the complete non-empty result:
SELECT frame
FROM timeless_aggregate_frame(
'metrics', 'cpu_usage', '{"env":"prod"}', :t0, :t1, 'avg');The versioned little-endian TAF1 layout is:
"TAF1" | aggregate_kind:u8 | flags:u8=0 | reserved:u16=0 |
series_count:u32 | series_ids:i64[series_count] |
validity_bitmap:u8[ceil(series_count/8)] |
value_words:u64[series_count]
Aggregate kinds are avg=0, sum=1, min=2, max=3, and count=4.
Valid float words contain IEEE-754 bits. Count words are nonnegative SQLite
INTEGER values. A clear validity bit is SQL NULL, its word must be zero, and
count is never NULL. Empty series are omitted; if every series is empty the TVF
emits no row. Series order is unspecified and labels attach through
timeless_series.
Rust callers use
timeless_ext::query_frame::decode_aggregate_frame, which rejects unknown
versions, flags, reserved bits, nonzero bitmap padding,
non-canonical NULLs, invalid kinds, and inconsistent lengths.
series_count is limited to u32; the encoder uses checked host-size
arithmetic and also remains subject to SQLite's configured maximum BLOB size.
timeless_latest_frame likewise returns the complete non-empty
timeless_latest result in one row:
SELECT frame
FROM timeless_latest_frame(
'metrics', 'cpu_usage', '{"env":"prod"}', :t0, :t1);The versioned little-endian TLF1 layout is:
"TLF1" | series_count:u32 |
series_ids:i64[series_count] | timestamps:i64[series_count] |
validity_bitmap:u8[ceil(series_count/8)] |
value_bits:u64[series_count]
Only series with a point in the inclusive range appear. Timestamp and
duplicate-winner semantics are identical to the row TVF. A clear validity bit
represents the row interface's SQL NULL for a NaN value and requires a zero
word. Use timeless_ext::query_frame::decode_latest_frame. TLF1, like every packed query
format, is an additive result envelope and never appears in shadow tables or
replication-visible storage.
series_count is limited to u32; frame-size arithmetic is checked before
allocation and SQLite's configured maximum BLOB size remains the practical
upper bound.
All current metric query surfaces are float-only. The machine-readable check is:
SELECT json_extract(
timeless_capabilities(),
'$.signals.metrics.sample_types'
) AS sample_types,
json_extract(
timeless_capabilities(),
'$.signals.metrics.native_histograms'
) AS native_histograms;
-- ["float64"] | 0This declaration covers row reads, TRF1, TWB1, TRB1, named/resolved
batches, chunks, and rollups. Classic _bucket float series remain supported;
no current SQL column or packed frame represents a typed native histogram.
timeless_raw_frame accepts the same table, metric, matcher filter, and
inclusive bounds as timeless_raw_batches, but returns one row for the whole
non-empty result set:
SELECT frame
FROM timeless_raw_frame(
'metrics', 'cpu_usage', '{"env":"prod"}', :t0, :t1,
:max_work_points);The optional trailing max_work_points is an inclusive positive INTEGER cap
on conservative stored-chunk point counts plus buffered points. It is checked
before persisted payload reads. Exceeding it returns an error and no partial
frame; omit the argument for the backward-compatible unbounded call.
The versioned little-endian frame is:
"TRF1" | series_count:u32 | total_points:u64 |
series_ids:i64[series_count] | point_counts:u32[series_count] |
timestamps:i64[total_points] | value_bits:u64[total_points]
The point counts partition both point columns into consecutive per-series
slices. Empty series are omitted, timestamps retain the stable raw-query order
inside each slice, and IEEE-754 value bits are preserved. Series slice order
is unspecified, just like SQL rows without ORDER BY; use the IDs to attach
catalog labels. Reject unknown magic, a length inconsistent with either header
count, or point counts whose sum differs from total_points.
The row-oriented timeless_raw and one-row-per-series
timeless_raw_batches interfaces remain available. TRF1 is additive and
changes neither shadow-table storage nor replication-visible formats.
Every packed raw/batch call increments public, per-process
timeless_stats(:table) rows named raw_batch_query_count,
raw_batch_query_total_ns, raw_batch_query_series_considered,
raw_batch_query_candidate_chunks, raw_batch_query_payload_bytes_read,
raw_batch_query_decoded_points,
raw_batch_query_buffered_points_considered, and
raw_batch_query_returned_points. Candidate chunks are timestamp-pruned
persisted chunks; decoded points count every point decompressed from those
chunks, even when bounds later discard it. The counters are observability
only, reset on process reopen, and are not persisted or transaction state.
Packed window calls expose the parallel cumulative keys
window_batch_query_count, window_batch_query_total_ns,
window_batch_query_series_considered,
window_batch_query_candidate_chunks,
window_batch_query_payload_bytes_read,
window_batch_query_decoded_points,
window_batch_query_buffered_points_considered, and
window_batch_query_returned_points through timeless_stats(:table).
They count the input chunks/points considered by the reduction and the sparse
grid points produced before optional null fill. Like raw counters, they are
per-process observability—not durable storage or transaction state.
timeless_window_batches accepts the same arguments and returns the same
per-series grid as timeless_window, but crosses SQLite once per series rather
than once per grid point:
SELECT series_id, labels, buckets
FROM timeless_window_batches(
'metrics', 'cpu_usage', '{"env":"prod"}',
:t0, :t1, 60, 300, 'avg', NULL, :max_work_points);The optional argument after fill caps both conservative input points and the
maximum matched series × grid points output, independently and inclusively,
before chunk payloads are read. Bind NULL for fill to request the default
sparse form while supplying the limit. Omit both trailing arguments to retain
the original call. Zero, negative, NULL-as-a-supplied-limit, and non-integer
limits fail explicitly.
The avg window fold uses compensated summation for cancellation-prone
finite values and an incremental-mean fallback before the running sum would
overflow. The sum fold uses the same compensated addition but retains
infinite overflow as its result. count includes every stored float regardless
of its IEEE value. NaN, infinities, and signed zero remain IEEE values in the
packed frame. The min and max folds ignore an incoming NaN once they have an
ordered extremum, replace a leading NaN with the first numeric sample, retain
NaN for an all-NaN window, and preserve the first of equal signed zeros. These
are general direct-SQL reductions; PromQL parsing, label/name
policy, timestamps, limits, and envelopes remain in the Rust metrics API.
last_over_time maps to the existing timeless_grid last-sample kernel and
retains exact IEEE bits; pinned Prometheus also retains its metric name.
The buckets blob is versioned and little-endian:
"TWB1" | count:u32 | timestamps:i64[count] |
validity_bitmap:u8[ceil(count/8)] | value_bits:u64[count]
Validity bit i is bit (i % 8) of byte (i / 8). Sparse calls contain
only present points, so every bit is set. With trailing fill='null', every
grid timestamp is encoded and a clear bit represents SQL NULL; the matching
value slot is zero and must be ignored. Reject unknown magic and lengths that
do not match the count. The row-oriented TVF remains the convenient SQL form;
the packed form is for host-language and remote boundaries where row crossings
are measurable.
timeless_rollup_batches takes the table, metric, filter, resolution, and
inclusive query bounds used by timeless_rollup, but returns every aggregate
at once. One row is emitted per non-empty matched series:
SELECT series_id, labels, buckets
FROM timeless_rollup_batches(
'metrics', 'cpu_usage', '{"env":"prod"}', 300, :t0, :t1);The versioned little-endian blob is:
"TRB1" | count:u32 |
bucket_ts:i64[count] | count:u64[count] | avg_bits:u64[count] |
sum_bits:u64[count] | min_bits:u64[count] | max_bits:u64[count] |
last_ts:i64[count] | last_value_bits:u64[count]
avg is computed from the stored sum and count at read time, just as it is in
the row TVF. Count stays integer-exact instead of passing through SQLite REAL;
the float columns preserve their IEEE-754 bits. last_ts exposes the timestamp
used to choose the stored last value and lets direct users retain the complete
rollup contract. Reject unknown magic and any length other than 8 + count * 64 bytes. The on-disk rollup payload and replication-visible shadow rows are
unchanged; TRB1 is only the public query envelope. The row-oriented
timeless_rollup remains available for ordinary SQL and single-aggregate
queries.
Charting libraries want dense grids. Two ways to get one:
Native (preferred): the optional trailing fill argument on
timeless_grid and timeless_window — 'none' (default) or
'null':
-- every grid point emitted per matched series; value is NULL where
-- the lookback window is empty
SELECT labels, ts, value
FROM timeless_grid('metrics', 'cpu_usage', NULL, :t0, :t1, 60, 90, 'null');The per-series absence rule still holds: a series with no points on
the grid at all emits nothing, filled or not (matching the waist's
query_multi omission rule). Gap-fill is presentation mechanics only —
which points have values is decided by the same kernel either way.
Portable SQL alternative (single series; also useful for right-edge padding beyond the data):
SELECT gs.value AS ts, g.value
FROM generate_series(:t0, :t1, 60) gs
LEFT JOIN timeless_grid('metrics', 'cpu_usage', '{"host":"web-1"}',
:t0, :t1, 60, 90) g
ON g.ts = gs.value;When you need counter math over the raw vtab (e.g. a range that mixes filters the kernels don't express), the standard reset-adjustment rule in window functions:
WITH s AS (
SELECT ts, value,
LAG(value) OVER (PARTITION BY labels ORDER BY ts) AS prev
FROM metrics
WHERE name = 'requests_total' AND ts > :t0 AND ts <= :t1
)
SELECT SUM(CASE WHEN prev IS NULL THEN 0 -- first sample: no step
WHEN value >= prev THEN value - prev -- monotone step
ELSE value END) AS increase -- reset: counter restarted
FROM s;
-- rate = increase / (:t1 - :t0)This computes exactly what timeless_window(..., 'increase') computes
over the window (:t0, :t1] — §33 asserts the two agree. This is a useful
storage statistic, but neither it nor the native rate fold implements
Prometheus edge extrapolation and zero-point clamping. For exact float-series
PromQL rate, use executable recipe
SQL-PROM-029 or the Rust metrics
API. Exact last-two-sample PromQL irate is documented separately as
SQL-PROM-030, and exact
extrapolated PromQL increase as
SQL-PROM-031. Exact
extrapolated gauge delta is
SQL-PROM-032, and final-pair
idelta is SQL-PROM-033.
Timestamp-centered least-squares gauge deriv is
SQL-PROM-034; the Rust API
adds Prometheus's compensated and IEEE-exact arithmetic to that finite SQL
foundation.
Evaluation-time-anchored gauge predict_linear is
SQL-PROM-035; its
:horizon is measured in seconds from each outer evaluation timestamp.
Ordered float-transition changes is
SQL-PROM-036, including the
public row surface's explicit SQL-NULL representation of stored NaN.
Strict float-counter decrease resets is
SQL-PROM-037; it does not
relabel the extension's mechanical reset-adjusted increase/rate kernels.
Pinned Prometheus 3.13.2 classifies double_exponential_smoothing as an
experimental function and disables it by default. The stable Timeless tier
therefore rejects it explicitly. It is not a missing stable range reduction;
enabling it later requires a separately configured experimental API tier and
its own enabled-oracle contract.
Bounded instant-vector abs is
SQL-PROM-038; ordinary SQLite
is exact for finite values, infinities, and signed zero, while the Rust API
retains packed NaN fidelity and PromQL labels, types, limits, and envelopes.
Bounded ceil, floor, and nearest-multiple round use
SQL-PROM-039,
including Prometheus's exact tie and scalar-step arithmetic.
Bounded clamp, clamp_min, and clamp_max use
SQL-PROM-040;
the Rust API adds bit-exact NaN/signed-zero behavior and per-step scalar-bound
expressions to the ordinary finite SQL foundation.
Bounded sqrt, exp, ln, log2, and log10 use
SQL-PROM-041;
the Rust API preserves Prometheus NaN/infinity domain results that ordinary
SQLite reports as SQL NULL.
Bounded sgn is SQL-PROM-042;
ordinary SQL preserves row-visible signed zero and the packed Rust path
retains true NaN.
Bounded inverse trigonometric and hyperbolic functions use
SQL-PROM-043;
the Rust API supplies the packed IEEE/domain distinctions ordinary SQLite
reports as SQL NULL.
Bounded trigonometric and hyperbolic functions use
SQL-PROM-044
with the same honest SQL-NULL versus packed-IEEE boundary.
Degree/radian conversion and scalar π use
SQL-PROM-045,
including a direct SQL evaluation-time recipe for pi().
PromQL label replacement is API-owned composition over the same bounded
public results:
label_replace(http_requests_total, "region", "$1", "instance", "([^.]+)\\..*")
The regular expression is a full-string, dot-all RE2-family match. Missing
source labels are read as empty strings; a matching empty replacement removes
the destination, while a non-match leaves the complete series unchanged.
Numbered and named captures, __name__ as source or destination, and
Prometheus 3's nonempty UTF-8 destination-label scheme are supported. Invalid
regexes and an empty destination return an execution envelope. SQLite's
public JSON functions can set or remove a known constant label, but ordinary
SQLite has no portable RE2-compatible capture-and-expand operation. The
PQL-F09 matrix foundation is therefore honestly none, and the cookbook
does not claim a general SQL equivalent or add an extension primitive solely
for language syntax. Replacement expansion consumes the response byte budget
incrementally, before amplified destination strings can accumulate.
Ordered label joining is also bounded API composition:
label_join(http_requests_total, "node", "/", "service", "instance")
Source labels are read in argument order from the original series. Missing
labels contribute empty strings, duplicate sources remain duplicated, and
zero source labels are valid. An empty joined value removes the destination;
__name__ can be a source or destination. Values, timestamps, and the metric
name are otherwise preserved. An empty destination returns an execution
error. Direct SQLite/libSQL users can perform the same arbitrary-arity
operation with the parameterized public-JSON statement in
SQL-PROM-046. Joined
destination strings use the same incremental response byte budget.
PromQL absence is evaluated at every outer timestamp:
absent(up{job="api", instance=~"web-.*"})
The result is empty whenever any input series has a sample. Otherwise it is a
single value 1; range queries assemble one sparse series from absent steps.
Only unique, nonempty equality matchers from a direct (optionally
parenthesized) selector become output labels. __name__, regex, negative,
empty, duplicate, and composed-expression matchers do not. NaN is still a
present sample. The executable public-grid equivalent is
SQL-PROM-047.
Window absence uses the same output-label and sparse-result rules, but tests the exact PromQL range interval independently at every outer timestamp:
absent_over_time(up{job="api", instance=~"web-.*"}[5m])
Each window is open on the left and closed on the right. Any stored sample,
including NaN, makes that step present and therefore removes it from the
result. Direct range selectors derive their unique nonempty equality labels;
subquery inputs are composed in Rust and derive none. The implementation
reuses the shipped bounded present_over_time plan and then performs the
step-local absence inversion, so limits and cancellation cover both stages.
Direct SQLite/libSQL users have the executable public-raw anti-join in
SQL-PROM-048.
PromQL ordering is intentionally an instant-vector presentation operation:
sort(http_request_duration_seconds)
sort_desc(http_request_duration_seconds)
sort orders samples by ascending value and sort_desc by descending value;
both put NaN last. Labels, metric names, timestamps, IEEE values, and nested
expression label policy are preserved. Equal values have no PromQL ordering
promise, so Timeless uses canonical labels as a deterministic tie-break.
Range-query results remain label-ordered matrices rather than pretending one
series order can represent a different value ordering at every step. Direct
SQLite/libSQL callers can use the parameterized instant statement and range
matrix statement in
SQL-PROM-049.
PromQL's explicit evaluation-type conversions are also API composition:
scalar(process_resident_memory_bytes{instance="api-1"})
vector(2 + 3)
scalar(vector) returns the sole sample value at each evaluation step and
returns NaN when that step has zero or multiple samples. A sole stored NaN is
still NaN. vector(scalar) produces one nameless series with the scalar value
at every step. Both compose with other shipped expressions and retain their
distinct scalar/vector instant and range envelopes. Direct SQLite/libSQL users
can use the executable per-step cardinality and nameless-vector statements in
SQL-PROM-050.
Evaluation and sample time are distinct PromQL values:
time()
timestamp(up{job="api"})
time() is the current evaluation timestamp in Unix seconds and retains
subsecond range-grid precision. timestamp(direct_selector) returns the
selected stored sample timestamp as its value while the response sample stays
on the outer evaluation grid; offset and @ alter selection without moving
that response grid. Once a unary, function, binary, aggregation, or range node
creates a new sample, timestamp() reports that node's evaluation time.
timestamp removes the metric name and preserves all other labels, including
for a stored NaN. Direct SQLite/libSQL equivalents for both clocks are in
SQL-PROM-051.
UTC calendar extraction accepts an optional instant vector:
minute(process_start_time_seconds)
hour()
day_of_week(vector(0))
day_of_month(process_start_time_seconds)
With no argument, each function evaluates vector(time()). Finite fractional
Unix seconds truncate toward zero, Sunday is day zero, metric names are
removed, and other labels remain. NaN, either infinity, and out-of-range
values follow the pinned Prometheus maximum-Unix-second conversion rather
than becoming NaN. Direct
SQLite/libSQL users can use the parameterized UTC strftime foundation in
SQL-PROM-052,
with its explicitly documented SQLite calendar-range limitation.
The remaining stable UTC calendar fields use the same optional-vector and conversion contract:
day_of_year(process_start_time_seconds)
days_in_month(vector(1709208000))
month()
year(process_start_time_seconds)
Day-of-year and month are one-indexed, days_in_month follows Gregorian leap
years, and all four preserve non-name labels while removing the metric name.
The parameterized direct SQLite/libSQL foundation, including leap-year and
zero-argument forms, is
SQL-PROM-053.
Classic float histograms use their ordinary cumulative *_bucket series:
histogram_quantile(
0.95,
sum by (service, le) (rate(http_request_duration_seconds_bucket[5m]))
)
Buckets are grouped by every label except le while retaining the metric name
as an internal family discriminator; both le and the metric name are removed
from the result. Equal numeric bounds are coalesced, material decreases are
made monotonic, relative deltas below Prometheus's 1e-12 tolerance are
ignored, and ranks interpolate linearly. A missing +Inf bucket, fewer than
two bounds, or a zero total returns NaN. Invalid quantiles retain Prometheus's
NaN/infinity result behavior. Malformed or absent le series are excluded.
If distinct metric families would produce the same visible label set at one
step after name removal, evaluation fails instead of emitting an invalid
duplicate vector.
All bucket points and the scalar quantile expression count toward the bounded
work limit, and the operation composes in Rust without another storage read.
This function applies only to classic float bucket series. Native histograms
remain deferred until the extension has an explicit typed storage model.
Direct SQLite/libSQL users can execute the parameterized bounded-grid recipe
in
SQL-PROM-054,
including the documented distinction between its ordinary-SQL foundation and
the API's complete Prometheus float/tolerance behavior.
histogram_fraction(lower, upper, buckets) estimates the fraction of classic
histogram observations between two scalar bounds:
histogram_fraction(
0.1,
0.5,
sum by (service, le) (rate(http_request_duration_seconds_bucket[5m]))
)
The pinned Prometheus classic-bucket algorithm coalesces equal numeric bounds,
requires a +Inf total, interpolates inside finite buckets, treats natural
zero and infinite-width buckets specially, and does not apply
histogram_quantile's monotonicity repair. Bounds may be scalar expressions
evaluated per step; inverted bounds return zero, zero totals and missing
+Inf return NaN, and output labels omit the metric name and le. Metric
families remain distinct internally, so a post-name-removal collision fails
instead of emitting duplicate label sets. Every selected bucket and scalar
bound is charged to cumulative work and cancellation limits.
This support is for ordinary classic float *_bucket series only. Native
histograms still require a typed storage model. Direct SQLite/libSQL users can
use the bounded ordinary-SQL CDF foundation in
SQL-PROM-056.
Prometheus's five native-histogram scalar functions are stable, but they ignore float samples rather than coercing them:
histogram_avg(request_duration)
histogram_count(request_duration)
histogram_sum(request_duration)
histogram_stddev(request_duration)
histogram_stdvar(request_duration)
Current Timeless metric samples are all float64, so each expression evaluates
its child with the normal public reads, limits, and cancellation contract and
then returns an empty vector (or empty range matrix). This exact behavior also
applies to classic *_bucket, *_sum, and *_count series: their names do
not turn independent float series into one native-histogram sample. Producing
count/sum/average or estimated bucket variance remains deferred until
PQL-S22 supplies a versioned typed sample and public result model. There is
no honest direct-SQL equivalent for the function contract today; querying or
combining classic series with SQL is a different operation.
Prometheus 3.13.2 feature-gates start(), end(), step(), range(),
min_of, max_of, and histogram_quantiles; start_timestamp() is not a
PromQL function. The stable endpoint returns the pinned disabled/unknown
diagnostics. This does not affect the shipped selector modifiers
@ start() and @ end(). MetricsQL variants remain separately tracked and
are never enabled by silently broadening PromQL.
MetricsQL-only syntax is accepted only on the explicitly named Rust API routes:
GET /metricsql/api/v1/query?query=%28cpu_usage+%3E+90%29+default+0&time=1700100010
GET /metricsql/api/v1/query_range?query=cpu_usage+if+on%28host%29+host_up&start=1700100000&end=1700100060&step=10s
default fills a missing left value from the matching right value at the
same evaluation step. if retains a left value only while a matching right
value exists; ifnot retains it only while the right value does not exist.
Matching ignores the metric name by default and accepts on(...) and
ignoring(...). A scalar operand is a nameless vector, so a scalar RHS can
broadcast across left label sets. The contributing left series keeps its
labels and metric name. As in pinned VictoriaMetrics 1.148.0, a join modifier
on these set-style operators does not rewrite those labels.
This differs deliberately from the stable PromQL routes, which continue to
reject default, if, and ifnot. MetricsQL scalar instant expressions also
return a nameless vector rather than a PromQL scalar. Timeless retains its
normal JSON error policy: invalid input is HTTP 400 bad_data, while
VictoriaMetrics uses HTTP 422 with error type 422. Execution limits and
cancellation use the same bounded Rust reader path as PromQL.
The extension does not parse MetricsQL. Direct SQLite/libSQL users can express
the storage-visible mechanics with the executable public-grid statements in
SQL-MQL-001.
The API remains responsible for precedence, implicit scalar vectors, label
policy, response envelopes, limits, and cancellation.
The MetricsQL routes accept keep_metric_names after a supported transform,
rollup, or binary operation:
abs({__name__=~"cpu_usage|memory_usage"}) keep_metric_names
rate(http_requests_total[5m]) keep_metric_names
(cpu_usage / 100) keep_metric_names
sum(abs({__name__=~"cpu_usage|memory_usage"}) keep_metric_names)
This is an operation modifier. It retains each contributing input metric name
during evaluation; it does not guess or restore a name after the result has
already collapsed. Multiple input names can therefore remain distinct through
a transform. Default binary matching also includes the metric name while the
modifier is active. An explicit on(host) still matches only host, then
retains the left metric name in the result. This distinction matches pinned
VictoriaMetrics 1.148.0.
Bare selectors, unary expressions, and aggregations cannot carry the trailing modifier and fail explicitly. An aggregate may consume a nested modified operation, as in the final example, and then applies its normal nameless output policy. The stable PromQL routes reject the syntax. Limits, cancellation, GET/form-POST behavior, durability, and reopen use the existing bounded Rust execution path.
SQLite/libSQL does not need a new query primitive: direct users retain the
public metric identity as an ordinary selected column. The executable
SQL-MQL-002
recipe shows the exact form and the name-aware/on(...) join distinction.
The explicit MetricsQL routes support both the named and parenthesized union
forms plus alias:
union(cpu_usage, memory_usage)
(cpu_usage, memory_usage)
alias(cpu_usage, "host_cpu_usage")
sum(union(alias(cpu_usage, "cpu"), alias(memory_usage, "memory")))
union() returns an empty vector, union(q) returns q, and both named and
parenthesized lists accept a trailing comma. Every argument is evaluated as a
bounded existing query plan. The union retains the first complete time series
when later arguments have the same metric name and labels; it never merges
their samples. Result ordering is not an argument-order contract.
alias(q, "name") replaces __name__ on every returned series; an empty name
removes it. Alias does not silently choose among series that become identical
after renaming. A bare alias that creates duplicate output labelsets fails,
matching pinned VictoriaMetrics 1.148.0. Likewise, union(1, 2) fails because
both scalar arguments become duplicate nameless vectors. Invalid arity,
non-string alias names, and empty comma slots fail explicitly.
The union function name is case-insensitive. alias is a lowercase built-in
template in pinned VictoriaMetrics, so ALIAS(...) is unsupported and
Timeless preserves that distinction.
Union and alias compose beneath stable operators and aggregations, but are
accepted only by /metricsql/api/v1/query and
/metricsql/api/v1/query_range; the PromQL routes retain their original
parser and reject both forms. Child results, collision state, output bytes,
and cancellation are charged to the existing bounded Rust query envelope.
The extension does not parse either construct. Direct SQLite/libSQL users use
ordinary public-grid UNION ALL, project an alias as the metric-name column,
and select the lowest branch for duplicate complete labelsets. The executable
SQL-MQL-003 recipe
pins that behavior and explains the bare-alias collision check.
The explicit MetricsQL routes support bounded label transformation after any scalar or instant-vector expression:
label_set(cpu_usage, "environment", "production", "host", "rewritten")
label_del(cpu_usage, "pod", "instance")
label_set(cpu_usage, "__name__", "host_cpu_usage")
label_del(cpu_usage, "__name__")
label_set applies label/value pairs from left to right, so the last repeated
destination wins. An empty value deletes that label rather than retaining an
empty string. Both functions accept no label arguments as an identity
operation, ignore deletion of a missing label, and treat __name__ as the
metric name. A scalar input becomes a nameless instant vector before the label
operation, matching pinned VictoriaMetrics 1.148.0.
The function names are case-insensitive built-in transforms, in contrast to
VictoriaMetrics's lowercase-only alias template. Trailing commas and the
otherwise redundant keep_metric_names modifier are accepted. Multiple
functions compose in argument order beneath ordinary operations and
aggregations. Invalid pair counts, non-string names or values, and empty
function calls fail explicitly.
Transforming multiple source series to the same complete output labelset is an error; the API does not silently choose a winner. Generated label bytes, intermediate points, response bytes, and cancellation checks use the existing bounded Rust query envelope. GET and form-encoded POST, instant and range responses, flush, shutdown, and reopen share that path. The stable PromQL routes retain their parser and reject both functions.
The extension does not parse this syntax. Direct SQLite/libSQL users can
project the metric-name column and use standard json_set/json_remove over
public-grid labels. The executable
SQL-MQL-004
recipe pins empty-value deletion, name handling, JSON paths, ordering, types,
and the duplicate-output check.
On the explicitly named MetricsQL routes, every bare selector is an implicit
default_rollup:
cpu_usage
default_rollup(cpu_usage)
default_rollup(cpu_usage[30s])
The first two forms are equivalent. For a range query, Timeless infers each
series' scrape interval from the interpolated 0.6 quantile of its last 20
intervals and applies VictoriaMetrics's jitter allowance. The automatic
window is at least the request step. Bind max_lookback=30s on the MetricsQL
request to cap this inferred default window; it does not shorten the explicit
[30s] form. Every range remains open on the left and closed on the right.
Instant requests use the request step directly.
The MetricsQL routes also accept the established one-argument rollups without a bracketed selector range:
avg_over_time(cpu_usage)
FIRST_OVER_TIME(cpu_usage,)
rate(http_requests_total)
changes(build_state)
timestamp(cpu_usage)
Supported window-less names are avg_over_time, min_over_time,
max_over_time, sum_over_time, count_over_time, present_over_time,
stddev_over_time, stdvar_over_time, first_over_time, last_over_time,
rate, irate, increase, delta, idelta, deriv, changes, and
resets. Function names are case-insensitive and a trailing comma is valid.
Statistical functions use the request step as their window. rate, irate,
and deriv use the pinned adjustable-window behavior; increase, delta,
idelta, changes, and resets can consume the bounded previous sample.
Counter functions apply VictoriaMetrics reset correction before calculating
the result.
default_rollup, average, minimum, maximum, first, and last retain the input
metric name. Other rollups and timestamp remove it. A default_rollup of a
scalar is a nameless vector and all forms compose under ordinary MetricsQL
operators and aggregations. Invalid arity fails with HTTP 400 bad_data.
Stable /api/v1/query* endpoints remain PromQL-only: they reject bracketless
rollups and keep first_over_time behind Prometheus's experimental tier.
The packed public raw path distinguishes the exact Prometheus stale-NaN bits from an ordinary stored NaN. A stale marker ends the visible result at that step. Timeless preserves and returns an ordinary NaN; pinned VictoriaMetrics discards that series during ingestion, so this stronger storage-fidelity behavior is documented rather than presented as oracle equality.
Direct SQLite/libSQL users can run the bounded automatic finite-series
selection and step-window reductions in
SQL-MQL-005.
The Rust API retains language parsing, exact packed-NaN handling, carry-in and
reset composition, names, limits, cancellation, and HTTP envelopes; no
MetricsQL syntax enters the extension.
The explicit MetricsQL routes support four transformations over the complete instant-vector evaluation grid of the current request:
range_avg(cpu_usage)
range_min(cpu_usage)
range_max(cpu_usage)
range_sum(cpu_usage)
range_sum(cpu_usage * 2)
These are not moving windows. Each input series is evaluated from inclusive
start through inclusive end at step, reduced once, and its final value is
repeated at every requested timestamp. An instant request therefore reduces a
one-point grid. Scalars become nameless vectors, arbitrary shipped scalar or
instant-vector expressions compose as the argument, names are
case-insensitive, and one trailing comma is accepted.
range_avg uses VictoriaMetrics's incremental arithmetic. After the first
non-NaN value, every grid slot—including a missing slot—advances the average's
position denominator. range_sum uses ordinary binary64 addition rather than
the extension's compensated window sum. Minimum and maximum choose the later
operand when values compare equal. Leading NaNs/missing values are skipped;
later gaps carry the running value. After reduction, the last non-NaN running
value fills the complete output grid, including leading gaps. A series with no
non-NaN value is omitted.
All four functions remove __name__. Pinned VictoriaMetrics does this even
for range_avg(q) keep_metric_names, so Timeless preserves that upstream
quirk instead of pretending the modifier restores the name. If removing names
collapses two input identities, the request fails with HTTP 422 execution
rather than merging samples. Zero/extra arguments fail with HTTP 400
bad_data. The stable PromQL routes reject every range_* name.
VictoriaMetrics's Remote Write/query path normalizes the signed-zero oracle
fixture to positive zero. Timeless preserves the stronger stored binary64
contract: because the extrema rule chooses the later equal operand,
range_min and range_max return -0 when that later operand has its sign bit
set. NaN, infinity, maximum-float overflow avoidance, sparse grids, duplicate
outputs, work/result limits, cancellation, GET/POST, shutdown, and reopen are
all real-extension regressions.
The API performs one bounded child evaluation and no second storage read.
Direct SQLite/libSQL users can run the slot-indexed recursive equivalent over
any public input grid with
SQL-MQL-006.
MetricsQL parsing, arbitrary expression composition, implicit lookback,
collision policy, limits, cancellation, and HTTP result shaping remain Rust
API responsibilities; no extension syntax or storage format changed.
The explicit MetricsQL routes also support cumulative transforms over the current request grid:
running_avg(cpu_usage)
running_min(cpu_usage)
running_max(cpu_usage)
running_sum(cpu_usage)
running_sum(cpu_usage * 2)
Unlike range_*, these functions emit the running state at each evaluation
timestamp. For input 11, 13, 15, running_avg returns 11, 12, 13 and
running_sum returns 11, 24, 39. An instant request is a one-slot grid.
Scalars become nameless vectors, shipped scalar/vector expressions compose as
the argument, names are case-insensitive, and one trailing comma is accepted.
Leading missing/NaN slots emit nothing. After the first value, a missing or
stale slot emits the previous state; it still advances running_avg's slot
index, so 1, missing, 2 becomes 1, 1, 1.3333333333333333. If actual
arithmetic computes NaN, that timestamp is omitted instead of carrying the
prior value. Average uses VictoriaMetrics's incremental update, sum uses
ordinary binary64 addition, and minimum/maximum choose the later equal
operand.
Every running function removes __name__, including when followed by
keep_metric_names. Post-removal duplicate identities fail with HTTP 422
execution; invalid arity fails with HTTP 400 bad_data. Stable PromQL routes
reject all four function names. Timeless retains stored signed-zero bits, so
equal extrema expose the later operand's zero sign even though the
VictoriaMetrics Remote Write fixture normalizes both zero orders.
One bounded child evaluation supplies the complete grid; cumulative folding
adds no storage read. Direct SQLite/libSQL users can execute the recursive
public-grid equivalent in
SQL-MQL-007.
MetricsQL parsing, arbitrary expression composition, packed missing/NaN
behavior, collisions, limits, cancellation, and HTTP envelopes remain Rust
API responsibilities; no extension syntax or storage format changed.
The explicit MetricsQL routes resolve the i duration suffix against the
positive step parameter of the current request:
count_over_time(cpu_usage[5i])
max_over_time(vector(time())[5i:1i])
cpu_usage offset 5i
cpu_usage offset -1i-1s
rate(http_requests_total[0i])
rate(http_requests_total[0i:1i])
Each Ni component contributes N * request_step milliseconds. Decimal and
compound components are accumulated as binary64 and the complete duration is
then truncated toward zero to a signed 64-bit millisecond value. Overflow
saturates at the corresponding int64 limit. Duration suffixes other than an
uppercase standalone M are case-insensitive; uppercase M remains the
VictoriaMetrics numeric multiplier, while Ms is milliseconds. A minus on
the first offset component is inherited by later positive components, so
offset -1i-1s means -(request_step + 1s), not -request_step + 1s.
Direct selector windows, subquery windows and resolutions, and signed offsets
all use the same request-owned resolution. A zero subquery resolution becomes
one request step. For ordinary range reductions, a resolved zero window also
becomes one request step. For default_rollup, rate, irate, and deriv,
an explicit 0i remains the upstream automatic-window signal: direct
selectors and subqueries retain the scrape-cadence inference documented in
the preceding rollup section. A zero offset remains zero.
The lowering pass ignores quoted strings and comments while accepting
comments between the range/offset delimiter and the duration. It uses a
collision-checked internal marker so a legitimate explicit duration with the
same millisecond value cannot be mistaken for 0i. Bare i is invalid and
returns the pinned parser diagnostic. The stable PromQL endpoints continue to
reject all i durations; the syntax never leaks into the primary language
tier.
All forms retain existing work, result, response, deadline, cancellation,
GET/POST, flush, shutdown, and reopen contracts. The extension receives no
MetricsQL grammar or new storage primitive. Direct SQLite/libSQL users can
bind request-step multiplication into the existing public window, grid, and
subquery recipes in
SQL-MQL-009;
adaptive zero-window rollups reuse SQL-MQL-005.
The explicit MetricsQL routes expose three case-insensitive, zero-argument functions whose values come only from the current request:
start()
end()
step()
query_contract_cpu + (start() - start())
start() and end() return the range request's inclusive bounds as
floating-point Unix seconds. For an instant query, both equal its evaluation
timestamp. step() returns the positive request step in seconds, including a
subsecond step such as 1.5. Negative pre-epoch request bounds remain
negative. The values compose as scalar expressions; MetricsQL's established
scalar-to-vector behavior determines the outer result envelope.
Each function requires exactly zero arguments. start_timestamp() and
range() are not supported by pinned VictoriaMetrics 1.148.0 and fail
explicitly rather than being treated as aliases. Stable PromQL continues to
feature-gate its similarly named functions. Existing selector modifiers
@ start() and @ end() retain their stable PromQL meaning on both routes;
the MetricsQL context lowering does not reinterpret their direct form.
Pure context expressions perform no extension query. Composition with a
selector performs the selector's one existing bounded public read. All work,
result, response, deadline, cancellation, GET/POST, shutdown, and reopen
contracts remain unchanged. No MetricsQL grammar or request state enters the
extension. Direct SQLite/libSQL users can bind the same request context with
the executable
SQL-MQL-010
recipe.
The explicit MetricsQL routes support VictoriaMetrics' plural classic- histogram form, whose destination label comes first and whose bucket expression comes last:
histogram_quantiles("phi", 0.25, 0.75, http_request_duration_seconds_bucket)
histogram_quantiles("phi", (time() - start()) / 20, histogram_bucket)
histogram_quantiles("rank", 0.5, native_vmrange_histogram)
Every rank is a scalar expression evaluated across the request grid. The
output label uses the first rank value with VictoriaMetrics formatting (for
example 1e+06, 1e-05, NaN, and +Inf), while the rank itself may vary at
later steps. The destination replaces an existing label; "__name__" sets a
new metric name and "" creates an empty-name label. The source bucket-family
name is otherwise always removed, even with keep_metric_names.
Classic cumulative le buckets and VictoriaMetrics non-cumulative
vmrange="lower...upper" buckets are accepted. vmrange groups are converted
to cumulative bounds in bounded Rust memory before quantile evaluation. Equal
numeric bounds are summed, the first NaN count becomes zero, later NaNs and
decreases are clamped to the preceding count, and the last bucket supplies the
total even when +Inf is absent. Interpolation starts from zero even when the
first bound is negative. A rank below zero returns -Inf, one above one
returns +Inf, and a rank landing in the infinite bucket returns the last
finite bound. Zero totals, NaN ranks, and other computed NaN samples are
omitted, matching the pinned VictoriaMetrics response behavior.
The bucket expression executes once regardless of the number of ranks.
Duplicate rank labels or destination replacement that collapses two bucket
groups fail as duplicate output timeseries. Malformed bounds are ignored,
invalid argument types fail explicitly, and all existing work, result,
response, deadline, and cancellation limits apply. The stable PromQL routes
continue to reject this MetricsQL argument order; Prometheus' experimental
vector-first function remains a separate PQL-H03 disposition.
Timeless preserves ordinary stored NaN bucket bits that VictoriaMetrics Remote
Write drops. The query therefore repairs those stored counts according to the
pinned VictoriaMetrics source algorithm, an intentional stronger-fidelity
storage boundary. No histogram state, MetricsQL syntax, or private-table access
was added to the extension. Direct SQLite/libSQL users can apply the executable
SQL-MQL-012
multi-rank recipe over public cumulative buckets.
The MetricsQL routes explicitly reject the finite MQL-08 catalog:
label_keep, label_map, quantiles, distinct, increase_pure,
remove_resets, interpolate, keep_last_value, keep_next_value,
drop_common_labels, rate_over_sum, and parser-time WITH templates. The
error is HTTP 400 bad_data, names the normalized construct and one-indexed
source position, and occurs before any public extension read. For example:
invalid parameter "query": 1:1: MetricsQL construct "label_keep" is deferred; it requires an individual compatibility row with pinned VictoriaMetrics semantics
Names are case-insensitive. Identical text inside double-quoted, single-quoted, or backtick strings and line comments is not classified as a construct. The list is exactly the old Elixir parser's rejection catalog; none of those features had Elixir execution semantics to port, and the list does not mean “all remaining MetricsQL.” Pinned VictoriaMetrics accepts all twelve, but they span unrelated transform, aggregate, rollup, and template contracts.
There is deliberately no executable MQL-08 SQL recipe. A rejection catalog
has no result semantics, and SELECT ... WHERE 0 would incorrectly skip child
evaluation. A future individual row must document its own public SQL
equivalent—or explicitly retain Rust API ownership—together with exact
boundaries, missing/IEEE/name/label behavior, limits, cancellation, and pinned
oracle parity.
Successful PromQL responses add top-level warnings and/or infos only when
Prometheus would emit them. The shipped float-series cases are invalid
quantile ranks, sort/sort_desc on a range query, rate/increase applied
to a metric name without a conventional counter suffix, missing or malformed
classic-histogram le labels, and material histogram monotonicity repair.
Text and one-indexed line/column positions match the pinned query source,
including nested calls and subqueries. GET and form-encoded POST requests,
instant and range envelopes, and shutdown/reopen use the same contract.
Annotations are deterministically deduplicated by their Prometheus message.
Histogram repair observations merge their timestamp, bucket, maximum-delta,
and sample-count span. Each severity emits at most ten messages plus one
omission summary, and annotation bytes count against the same response-size
limit as data. An unaffected query or a counter-like function that produces
no sample omits both fields rather than returning empty arrays. These are API
language diagnostics; no annotation syntax or state is added to SQLite.
Prefer the native kernel only when its explicitly mechanical semantics are the desired contract; it decompresses once in the engine and ships grid points rather than raw samples over sqld/HTTP.
"Top 2 hosts by average CPU per minute" — ROW_NUMBER over a bucketed
aggregate (works on the raw vtab; substitute a timeless_window call
as the inner query for big ranges):
WITH b AS (
SELECT labels, (ts / 60) * 60 AS bucket_ts, AVG(value) AS v
FROM metrics
WHERE name = 'cpu_usage' AND ts >= :t0 AND ts <= :t1
GROUP BY labels, bucket_ts
),
r AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY bucket_ts ORDER BY v DESC) AS rn
FROM b
)
SELECT bucket_ts, labels, v FROM r WHERE rn <= 2
ORDER BY bucket_ts, rn;Error ratio = two kernel calls joined on (labels, ts) — grids from
the same (start, stop, step) land on identical grid points, which is
what makes this join safe:
SELECT e.ts, e.labels, e.value / r.value AS error_ratio
FROM timeless_grid('metrics', 'errors_total', NULL, :t0, :t1, 60, 90) e
JOIN timeless_grid('metrics', 'requests_total', NULL, :t0, :t1, 60, 90) r
ON r.labels = e.labels AND r.ts = e.ts;(labels is canonical JSON — sorted keys, minimal escaping — so string
equality is label-set equality.)
The engine never decides what an outlier is; you say so in SQL. Three escalating options:
Trimmed mean (kernel): drop a fixed fraction from each tail —
timeless_window(..., 'tavg:5').
IQR fences (Tukey): quartiles from the exact-percentile kernel, cut
raw samples outside [q1 − 1.5·IQR, q3 + 1.5·IQR]:
WITH fences AS (
SELECT (SELECT value FROM timeless_window('metrics', 'latency', NULL,
:t1, :t1, 1, :t1 - :t0, 'p25')) AS q1,
(SELECT value FROM timeless_window('metrics', 'latency', NULL,
:t1, :t1, 1, :t1 - :t0, 'p75')) AS q3
)
SELECT AVG(value) AS robust_avg
FROM metrics, fences
WHERE name = 'latency' AND ts > :t0 AND ts <= :t1
AND value BETWEEN q1 - 1.5 * (q3 - q1) AND q3 + 1.5 * (q3 - q1);σ-based (2-sigma): population stddev in plain SQL:
WITH stats AS (
SELECT AVG(value) AS mu,
sqrt(AVG(value * value) - AVG(value) * AVG(value)) AS sigma
FROM metrics WHERE name = 'latency' AND ts > :t0 AND ts <= :t1
)
SELECT AVG(value) AS robust_avg
FROM metrics, stats
WHERE name = 'latency' AND ts > :t0 AND ts <= :t1
AND ABS(value - mu) <= 2 * sigma;Caveat worth knowing: with tiny samples a single huge outlier inflates
σ enough to mask itself — IQR fences and tavg:N are the sturdier
tools there.