Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/olive-pugs-marry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
"cnosdb-client": minor
---

Add `client.queryTable()`, which returns a result's columns alongside its rows. It requests CSV, the only CnosDB response format that carries column names in their true order, so the columns come back in the order the statement selected them and every row has exactly one value per column. On CnosDB 2.4.3 the columns survive an empty result, so a table with no matching rows can still be rendered with its headings; 2.4.1 returns an empty body instead and reports no columns.

This matters because the JSON format used by `query()` sorts keys alphabetically and omits any column that is NULL for a given row, which makes row objects differ in shape and hides nulls entirely. Both behaviours are now documented in `docs/compatibility.md`.

Values are returned as raw strings, because CnosDB sends no column types over HTTP in any response format; converting them would mean guessing.

Also exports the `Compression` type from the package root, which was added as a client option in 0.2.0 but was not importable.
35 changes: 35 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,40 @@ The statement is sent verbatim. The client does not rewrite, interpolate, or
retry it. Statements that return no rows (such as DDL) resolve to `undefined`;
use `execute()` for those.

## Querying with column metadata

`query()` returns JSON objects, which is convenient but loses two things: CnosDB
sorts the keys alphabetically rather than by the order you selected, and it
**omits any column that is NULL for that row**, so row objects can differ in
shape from one row to the next.

`queryTable()` asks for CSV instead, the only format that carries column names
and their order:

```ts
const { columns, rows } = await client.queryTable(
"SELECT v, city FROM weather",
);
// columns: ["v", "city"] — the order you asked for
// rows: [["1.5", "Pokhara"]] — always one value per column
```

Every row has exactly one value per column, so a NULL stays visible as an empty
string rather than vanishing.

On CnosDB 2.4.3 the columns are reported even when no rows match, so an empty
result can still be rendered with its headings. Do not rely on that below 2.4.3:
2.4.1 returns an empty body for an empty result, and `columns` is then also
empty. `rows` is empty either way.

Values are raw strings. CnosDB sends no column types over HTTP in any response
format, so converting them would mean guessing, and a wrong guess on a large
integer or a timestamp is worse than an honest string. Convert what you need at
the call site.

One ambiguity is unavoidable: CnosDB renders both NULL and an empty string as an
empty field, so the two cannot be told apart in a `queryTable()` result.

## Executing SQL

```ts
Expand Down Expand Up @@ -330,6 +364,7 @@ new CnosDBClient(options: CnosDBClientOptions)

client.ping(options?: RequestOptions): Promise<PingResult>
client.query<T>(statement: string, options?: QueryOptions): Promise<T>
client.queryTable(statement: string, options?: QueryOptions): Promise<QueryTable>
client.execute(statement: string, options?: QueryOptions): Promise<void>
client.writeLineProtocol(data: string, options?: WriteOptions): Promise<void>
client.writePoints(points: Point | readonly Point[], options?: WriteOptions): Promise<void>
Expand Down
5 changes: 4 additions & 1 deletion docs/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,10 @@ Authentication is HTTP Basic. SQL is sent as the request body with
Observed on CnosDB 2.4.3 and encoded in the tests:

- `GET /api/v1/ping` returns `{"version": "...", "status": "healthy"}` and needs no authentication.
- A successful `SELECT` with `Accept: application/json` returns a JSON array of row objects.
- A successful `SELECT` with `Accept: application/json` returns a JSON array of row objects, with two caveats worth knowing. Keys are sorted **alphabetically**, not in the order the statement selected them, so `SELECT v, city` returns `{"city": ..., "v": ...}`. A column that is NULL for a row is **omitted from that row's object** entirely, so row objects can differ in shape and a NULL cannot be distinguished from an absent column.
- `Accept: application/csv` and `text/csv` return a header row followed by data rows. This is the only format that carries column names in their true order, and it emits every column for every row, so it is what `queryTable()` uses. On 2.4.3, an empty result set still returns the header row, whereas the JSON format returns a completely empty body. **This differs by version:** 2.4.1 returns an empty body for an empty CSV result too, so `queryTable()` reports no columns there. The ping string cannot be used to branch on this, since 2.4.1 identifies itself as 2.4.0. Fields are quoted per RFC 4180, with doubled quotes for a literal quote. Both NULL and an empty string render as an empty field and cannot be told apart.
- `Accept: application/nd-json` returns newline-delimited JSON objects. `application/x-ndjson` is rejected with `040005`.
- No response format carries column **types**.
- DDL such as `CREATE DATABASE` returns HTTP 200 with an **empty body**. `query()` therefore resolves to `undefined` for such statements; use `execute()` instead.
- Invalid SQL returns HTTP **422** with a JSON body such as `{"error_code":"030019","error_message":"Table not found: ..."}`. This maps to `CnosDBRequestError`.
- CnosDB **never returns HTTP 401**. It reuses 422 for nearly every application failure and distinguishes them only by `error_code`, so the client classifies errors on that code rather than on the status. The 401 mapping is retained for proxies that do use it.
Expand Down
36 changes: 36 additions & 0 deletions src/client/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@ import {
Transport,
} from "../http/index.js";
import { serializePoints } from "../line-protocol/index.js";
import { parseCsv } from "../csv/index.js";
import type {
CnosDBClientOptions,
Compression,
Point,
PingResult,
QueryOptions,
QueryTable,
RequestOptions,
TimePrecision,
WriteOptions,
Expand Down Expand Up @@ -171,6 +173,40 @@ export class CnosDBClient {
return result as T;
}

/**
* Executes a SQL statement and returns its columns alongside raw row values.
*
* Use this when the columns matter: rendering a table, exporting data, or
* running a statement whose shape is not known in advance. It asks CnosDB
* for CSV, which is the only response format that carries the column names
* and their order; the JSON format sorts keys alphabetically and omits any
* column that is NULL for a given row.
*
* Values are returned as raw strings, because CnosDB sends no column types
* over HTTP. See {@link QueryTable} for what that implies.
*/
async queryTable(
statement: string,
options: QueryOptions = {},
): Promise<QueryTable> {
const sql = requireStatement(statement);
const body = await this.#transport.requestText({
method: "POST",
path: SQL_PATH,
searchParams: this.#sqlParams(options),
body: sql,
contentType: "text/plain; charset=utf-8",
accept: "application/csv",
...requestControls(options),
});

const parsed = parseCsv(body);
// A statement with no result set at all, such as DDL, returns an empty
// body rather than a header row.
const [columns, ...rows] = parsed;
return { columns: columns ?? [], rows };
}

/**
* Executes a SQL statement whose result rows are not needed, such as DDL.
* Any 2xx response counts as success and the body is discarded.
Expand Down
1 change: 1 addition & 0 deletions src/csv/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export { parseCsv } from "./parse.js";
81 changes: 81 additions & 0 deletions src/csv/parse.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
/**
* Minimal RFC 4180 parser for CnosDB's CSV responses.
*
* A hand-written parser rather than a dependency: the grammar is small, the
* input comes from one known producer, and a parser is easier to audit than an
* extra supply-chain entry in a client whose whole job is talking to one
* server.
*
* @internal
*/
export function parseCsv(input: string): string[][] {
const rows: string[][] = [];
let row: string[] = [];
let field = "";
let quoted = false;
let fieldStarted = false;

const endField = (): void => {
row.push(field);
field = "";
fieldStarted = false;
};

const endRow = (): void => {
endField();
rows.push(row);
row = [];
};

for (let index = 0; index < input.length; index += 1) {
const char = input[index] as string;

if (quoted) {
if (char !== '"') {
field += char;
continue;
}
// A doubled quote inside a quoted field is one literal quote.
if (input[index + 1] === '"') {
field += '"';
index += 1;
continue;
}
quoted = false;
continue;
}

if (char === '"' && !fieldStarted) {
quoted = true;
fieldStarted = true;
continue;
}

if (char === ",") {
endField();
continue;
}

if (char === "\r") {
// Tolerate both CRLF and a bare CR as a row terminator.
if (input[index + 1] === "\n") index += 1;
endRow();
continue;
}

if (char === "\n") {
endRow();
continue;
}

field += char;
fieldStarted = true;
}

// A trailing newline ends the last row rather than starting an empty one.
if (field.length > 0 || fieldStarted || row.length > 0 || quoted) {
endRow();
}

return rows;
}
6 changes: 6 additions & 0 deletions src/http/transport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,12 @@ export class Transport {
}
}

/** Performs a request and returns the response body as text. */
async requestText(request: TransportRequest): Promise<string> {
const response = await this.request(request);
return this.#readSuccessBody(response, request);
}

/**
* Performs a request and discards the response body, ensuring the
* underlying connection is not left half-read.
Expand Down
2 changes: 2 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@ export {
export type { CnosDBErrorOptions } from "./errors/index.js";
export type {
CnosDBClientOptions,
Compression,
FetchLike,
PingResult,
Point,
PointFieldValue,
QueryOptions,
QueryTable,
RequestOptions,
TimePrecision,
WriteOptions,
Expand Down
1 change: 1 addition & 0 deletions src/types/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,5 @@ export type {
WriteOptions,
} from "./request-options.js";
export type { PingResult } from "./ping.js";
export type { QueryTable } from "./query-table.js";
export type { Point, PointFieldValue } from "./point.js";
28 changes: 28 additions & 0 deletions src/types/query-table.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/**
* A query result with its columns, in the order the server returned them.
*
* This is the shape to reach for when the columns matter: rendering a table,
* exporting to a file, or handling a statement whose shape is not known ahead
* of time. Use {@link CnosDBClient.query} instead when you know the columns and
* want convenient JavaScript values.
*/
export interface QueryTable {
/**
* Column names in the order the server produced them.
*
* On CnosDB 2.4.3 these survive an empty result, so a table with no matching
* rows can still be rendered with its headings. Older servers, including
* 2.4.1, return an empty body instead and this is then empty too.
*/
readonly columns: readonly string[];

/**
* Rows as raw field strings, aligned with {@link columns}.
*
* Values are strings because CnosDB sends no column types over HTTP; nothing
* is converted, so nothing is guessed. A NULL arrives as an empty string and
* is indistinguishable from an empty string value, which is a limitation of
* the server's CSV output rather than a choice made here.
*/
readonly rows: readonly (readonly string[])[];
}
82 changes: 82 additions & 0 deletions tests/integration/client.integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,88 @@ describe("writes and queries", () => {
});
});

describe("queryTable", () => {
const table = "table_shape";

it("returns columns in the order the statement asked for", async () => {
await client.writePoints(
{
measurement: table,
tags: { city: "Pokhara" },
fields: { v: 1.5, n: 7n },
timestamp: Date.now(),
},
{ database, precision: "ms" },
);

const result = await client.queryTable(
`SELECT v, city FROM ${table} LIMIT 1`,
{ database },
);

// The JSON endpoint sorts keys alphabetically and would report city first,
// which is why this method exists.
expect(result.columns).toEqual(["v", "city"]);
expect(result.rows[0]).toHaveLength(2);
});

it("returns no rows for an empty result set", async () => {
const result = await client.queryTable(
`SELECT v, city FROM ${table} WHERE city = 'nowhere-at-all'`,
{ database },
);

expect(result.rows).toEqual([]);
// Whether the columns survive an empty result is version-dependent: 2.4.3
// sends the header row, 2.4.1 sends an empty body. Both are accepted here
// because the ping version cannot tell those releases apart — 2.4.1
// reports itself as 2.4.0.
expect([[], ["v", "city"]]).toContainEqual(result.columns);
});

it("keeps a NULL column aligned instead of dropping it", async () => {
// The same row read as JSON omits the null key entirely, so the object
// shape silently changes between rows.
await client.writePoints(
{
measurement: table,
tags: { city: "Lalitpur" },
fields: { v: 2.5 },
timestamp: Date.now(),
},
{ database, precision: "ms" },
);

const result = await client.queryTable(
`SELECT city, v, n FROM ${table} WHERE city = 'Lalitpur'`,
{ database },
);

expect(result.columns).toEqual(["city", "v", "n"]);
for (const row of result.rows) {
expect(row).toHaveLength(3);
}
});

it("decodes a value containing a comma and a quote", async () => {
await client.writePoints(
{
measurement: "table_escaping",
tags: { kind: "csv" },
fields: { s: 'a,b"c' },
timestamp: Date.now(),
},
{ database, precision: "ms" },
);

const result = await client.queryTable("SELECT s FROM table_escaping", {
database,
});

expect(result.rows[0]).toEqual(['a,b"c']);
});
});

describe("cancellation", () => {
it("rejects with an abort error when the caller cancels", async () => {
const controller = new AbortController();
Expand Down
Loading