diff --git a/docs/ERRORS.md b/docs/ERRORS.md new file mode 100644 index 0000000..2fc69b0 --- /dev/null +++ b/docs/ERRORS.md @@ -0,0 +1,42 @@ +# Error-code reference + +This reference lists every code declared in `src/core/errors.ts`. The `check:errors` validator +compares the table with the `ERROR_CODES` registry and scans `src/` for +`finding(ERROR_CODES., ...)` call sites. The code list and emission status therefore stay +aligned with the implementation; the cause and smallest-fix descriptions remain reviewed +documentation. + +`emitted` means that the current source has at least one `ERROR_CODES.` call site. `defined +but not emitted` is retained below because the code is part of the exported registry, but no +current input path produces it. A warning can still appear in a successful receipt; it is not an +indication that data should be changed merely to remove the warning. + +When a fix concerns observed responses or research provenance, use the verified source value or +actual citation. Do not invent provenance, drop rows or items, or alter observations only to make +an audit quieter. + +| Code | Emission | Source locations | Cause | Smallest input fix | +| --- | --- | --- | --- | --- | +| `not_object` | emitted | `src/core/parse.ts` | A parser received a primitive, `null`, or array where a JSON object was required. | Replace the value at the reported path with an object, then provide that object's required fields. | +| `invalid_schema_version` | emitted | `src/core/parse.ts` | `schemaVersion` does not equal the schema version required by the parser. | Set `schemaVersion` to the exact version named in the finding message. | +| `missing_field` | emitted | `src/bundle.ts`, `src/core/parse.ts`, `src/dataset/index.ts`, `src/gap-map/index.ts` | A required field or relationship is missing; dataset audits also use this code for missing indicator cells as a warning. | Add the required field or mapping. For a missing observed cell, provide the verified response or leave the audit warning rather than inventing a value. | +| `invalid_type` | emitted | `src/core/parse.ts`, `src/dataset/index.ts`, `src/gap-map/index.ts`, `src/instrument/index.ts`, `src/measurement/index.ts`, `src/recode/index.ts`, `src/topic-card/index.ts` | A value has the wrong JSON type or an unknown field was supplied; the dataset audit also uses this code for straight-line warnings. | Use the type required by the finding or remove the unknown field. A straight-line warning has no data-edit fix; review the observations instead. | +| `invalid_identifier` | emitted | `src/core/parse.ts`, `src/instrument/index.ts` | A construct or item code fails the code pattern, a column name fails the column-name pattern, or an item code does not use its construct prefix as a warning. | Use a valid identifier and, for the prefix warning, rename the item with the construct prefix while preserving its meaning. | +| `invalid_enum` | emitted | `src/core/parse.ts`, `src/recode/index.ts` | An enum field contains a value outside the closed vocabulary declared by the parser. | Replace it with one of the allowed values listed in the finding message. | +| `empty_collection` | emitted | `src/core/parse.ts`, `src/instrument/index.ts`, `src/measurement/index.ts` | A required collection is absent, not an array, or empty; measurement compilation also uses this code as a warning when structural paths are absent. | Provide at least one valid member when the collection is required. For a measurement-only warning, add only a real intended hypothesis or accept the warning. | +| `duplicate_code` | emitted | `src/dataset/index.ts`, `src/instrument/index.ts`, `src/topic-card/index.ts` | A construct code, item code, hypothesis ID, anchor value, or respondent ID is repeated. | Correct the duplicate identifier or anchor using the real distinct value; do not duplicate or delete research records just to silence the finding. | +| `duplicate_column` | emitted | `src/dataset/index.ts` | Two dataset column definitions have the same name. | Rename one column to a new valid name or remove the redundant definition after confirming the source mapping. | +| `invalid_scale` | emitted | `src/dataset/index.ts`, `src/gap-map/index.ts`, `src/instrument/index.ts` | A declared scale is invalid, an observed scale hint is not made of integers, or an observed response is outside the instrument's Likert bounds. | Declare an allowed scale with `min < max`, correct a scale hint to the verified integers, or correct the response only when the source record proves it was entered incorrectly. | +| `invalid_anchor` | emitted | `src/instrument/index.ts` | A scale anchor is not an integer or falls outside the declared scale. | Use an integer anchor within the declared `min` and `max` values. | +| `cited_without_source` | emitted | `src/instrument/index.ts` | An item is marked `cited` without a non-empty `source.citation`. | Add the actual citation, or change the item status to `demonstration` when no citation exists. | +| `reverse_scale_unsupported` | emitted | `src/instrument/index.ts` | A scale with anchors does not include both scale endpoints, so its reverse mapping is not supported. | Add verified anchors for both declared endpoints, or omit the optional anchors when they are not needed. | +| `unknown_construct` | emitted | `src/recode/index.ts`, `src/topic-card/index.ts` | A topic card or recode operation refers to a construct absent from the instrument. | Reference an existing construct code or add the real construct definition to the instrument. | +| `unknown_item` | emitted | `src/dataset/index.ts`, `src/recode/index.ts` | A dataset indicator or reverse-score operation refers to an item absent from the instrument; audits also warn when an instrument item has no mapped indicator column. | Correct the item mapping to an existing item or add the real indicator column when it exists in the source dataset. | +| `orphan_hypothesis` | emitted | `src/topic-card/index.ts` | A hypothesis has the same source and target, or refers to a construct outside the selected instrument constructs. | Reference two distinct existing construct codes and include both in `constructCodes`. | +| `row_count_mismatch` | emitted | `src/dataset/index.ts` | `rowCount` is not equal to the number of entries in `rows`. | Set `rowCount` to the exact current `rows.length`; do not drop or add rows as a validation shortcut. | +| `unknown_column` | emitted | `src/dataset/index.ts`, `src/recode/index.ts` | A row contains an undeclared column, or a recode operation refers to a missing source or indicator column. | Declare the actual column and its mapping, or change the operation to an existing column. | +| `column_exists` | emitted | `src/recode/index.ts` | A recode target already exists in the dataset or in an earlier operation. | Choose a new valid `as` column name that is not already present. | +| `non_numeric` | emitted | `src/dataset/index.ts`, `src/recode/index.ts` | A dataset audit or numeric recode operation encountered a non-numeric indicator value. | Supply the verified numeric response or use `null` for a genuinely missing response; do not coerce arbitrary text. | +| `empty_operations` | defined but not emitted | `src/core/errors.ts` registry only | The code is exported in the registry, but the current source has no `finding(ERROR_CODES.empty_operations, ...)` call site. Empty recode operations currently use `empty_collection`. | No current input triggers this code. Do not document it as a runtime result unless a source call site is added and this table is updated. | +| `under_identified` | emitted | `src/measurement/index.ts` | A measurement construct has fewer items than the mode-and-kind heuristic minimum; this is a warning unless `strict` is true. | Add enough real indicators to meet the applicable minimum: reliability-only 2, PLS reflective 2 or formative 1, CB-SEM reflective 3 or formative 2. This does not prove model fit. | +| `invalid_receipt` | emitted | `src/core/receipt.ts` | A receipt has an invalid shape, status, digest, compiler field, finding, count, or digest mismatch with its input or artifact. | Regenerate the receipt from the exact input and artifact produced together; do not hand-edit digest fields. | diff --git a/package.json b/package.json index 7bcffbf..46465ad 100644 --- a/package.json +++ b/package.json @@ -52,9 +52,10 @@ "test:watch": "vitest", "check:pack": "node scripts/check-pack.mjs", "check:schema": "node scripts/check-schema.mjs", + "check:errors": "node scripts/check-errors.mjs", "check:secrets": "node scripts/check-secrets.mjs", "check:smoke": "node scripts/cli-smoke.mjs", - "verify": "npm run typecheck && npm run build && npm run test && npm run check:schema && npm run check:pack && npm run check:secrets && npm run check:smoke" + "verify": "npm run typecheck && npm run build && npm run test && npm run check:schema && npm run check:errors && npm run check:pack && npm run check:secrets && npm run check:smoke" }, "devDependencies": { "@types/node": "^22.15.3", diff --git a/scripts/check-errors.mjs b/scripts/check-errors.mjs new file mode 100644 index 0000000..ea509e8 --- /dev/null +++ b/scripts/check-errors.mjs @@ -0,0 +1,216 @@ +#!/usr/bin/env node + +import { readdir, readFile, stat } from "node:fs/promises"; +import { join, relative, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import * as ts from "typescript"; + +const root = fileURLToPath(new URL("..", import.meta.url)); + +async function listSourceFiles(directory) { + const entries = await readdir(directory, { withFileTypes: true }); + const files = []; + for (const entry of entries.sort((left, right) => left.name.localeCompare(right.name))) { + const path = join(directory, entry.name); + if (entry.isDirectory()) { + files.push(...(await listSourceFiles(path))); + } else if (entry.name.endsWith(".ts")) { + files.push(path); + } + } + return files; +} + +function tableCells(line) { + const cells = line.trim().split("|"); + if (cells.length < 3 || cells[0] !== "" || cells.at(-1) !== "") { + return undefined; + } + return cells.slice(1, -1).map((cell) => cell.trim()); +} + +export function findFindingCodes(source, fileName = "source.ts") { + const sourceFile = ts.createSourceFile(fileName, source, ts.ScriptTarget.Latest, true, ts.ScriptKind.TS); + const codes = []; + + function visit(node) { + if ( + ts.isCallExpression(node) && + ts.isIdentifier(node.expression) && + node.expression.text === "finding" + ) { + const firstArgument = node.arguments[0]; + if ( + firstArgument && + ts.isPropertyAccessExpression(firstArgument) && + ts.isIdentifier(firstArgument.expression) && + firstArgument.expression.text === "ERROR_CODES" && + ts.isIdentifier(firstArgument.name) + ) { + codes.push(firstArgument.name.text); + } + } + ts.forEachChild(node, visit); + } + + visit(sourceFile); + return codes; +} + +export function extractSourceLocationPaths(sourceCell) { + return [...sourceCell.matchAll(/`(src\/[^`]+)`/g)].map(([, path]) => path); +} + +async function isValidSourceLocation(projectRoot, sourcePath) { + if (!sourcePath.startsWith("src/") || sourcePath.includes("..") || !sourcePath.endsWith(".ts")) { + return false; + } + const absolutePath = join(projectRoot, sourcePath); + if (relative(projectRoot, absolutePath) !== sourcePath) { + return false; + } + try { + return (await stat(absolutePath)).isFile(); + } catch { + return false; + } +} + +export async function checkErrorReference(projectRoot = root) { + const sourceRoot = join(projectRoot, "src"); + const registryPath = join(sourceRoot, "core", "errors.ts"); + const documentationPath = join(projectRoot, "docs", "ERRORS.md"); + const errors = []; + + const registrySource = await readFile(registryPath, "utf8"); + const registryMatch = registrySource.match(/export const ERROR_CODES = \{([\s\S]*?)\} as const;/); + if (!registryMatch) { + errors.push(`Could not find the ERROR_CODES registry in ${relative(projectRoot, registryPath)}.`); + return { errors, registrySize: 0, emittedCount: 0 }; + } + + const registryEntries = [...registryMatch[1].matchAll(/^\s{2}([A-Za-z_][A-Za-z0-9_]*):\s*"([^"]+)",?\s*$/gm)].map( + ([, key, value]) => ({ key, value }), + ); + const registryByCode = new Map(); + for (const entry of registryEntries) { + if (registryByCode.has(entry.value)) { + errors.push(`Duplicate ERROR_CODES value '${entry.value}'.`); + } + registryByCode.set(entry.value, entry); + } + if (registryEntries.length === 0) { + errors.push("ERROR_CODES registry contains no parseable entries."); + } + + const usageByCode = new Map([...registryByCode.keys()].map((code) => [code, new Set()])); + for (const path of await listSourceFiles(sourceRoot)) { + const source = await readFile(path, "utf8"); + for (const key of findFindingCodes(source, path)) { + const entry = registryEntries.find((candidate) => candidate.key === key); + if (!entry) { + errors.push(`Source emits ERROR_CODES.${key}, but it is absent from the registry.`); + continue; + } + usageByCode.get(entry.value).add(relative(projectRoot, path)); + } + } + + const documentation = await readFile(documentationPath, "utf8"); + const lines = documentation.split(/\r?\n/); + const header = "| Code | Emission | Source locations | Cause | Smallest input fix |"; + const headerIndex = lines.indexOf(header); + if (headerIndex === -1 || lines[headerIndex + 1] !== "| --- | --- | --- | --- | --- |") { + errors.push("docs/ERRORS.md must contain the expected five-column error table."); + } + + const rows = []; + if (headerIndex !== -1) { + for (const line of lines.slice(headerIndex + 2)) { + if (!line.startsWith("|")) { + if (rows.length > 0) { + break; + } + continue; + } + const cells = tableCells(line); + if (!cells || cells.length !== 5) { + errors.push(`Malformed error table row: ${line}`); + continue; + } + rows.push({ + code: cells[0].replaceAll("`", ""), + emission: cells[1], + source: cells[2], + cause: cells[3], + fix: cells[4], + }); + } + } + + const rowsByCode = new Map(); + for (const row of rows) { + if (rowsByCode.has(row.code)) { + errors.push(`docs/ERRORS.md lists '${row.code}' more than once.`); + } + rowsByCode.set(row.code, row); + if (!row.source || !row.cause || !row.fix) { + errors.push(`docs/ERRORS.md row '${row.code}' must include source, cause, and smallest input fix.`); + } + const sourcePaths = extractSourceLocationPaths(row.source); + if (sourcePaths.length === 0) { + errors.push(`docs/ERRORS.md row '${row.code}' must include at least one backticked src/*.ts source location.`); + } + for (const sourcePath of sourcePaths) { + if (!(await isValidSourceLocation(projectRoot, sourcePath))) { + errors.push(`docs/ERRORS.md row '${row.code}' references missing or invalid source path '${sourcePath}'.`); + } + } + } + + for (const code of registryByCode.keys()) { + const row = rowsByCode.get(code); + if (!row) { + errors.push(`docs/ERRORS.md is missing registry code '${code}'.`); + continue; + } + const expectedEmission = usageByCode.get(code).size > 0 ? "emitted" : "defined but not emitted"; + if (row.emission !== expectedEmission) { + errors.push(`Code '${code}' is marked '${row.emission}', expected '${expectedEmission}'.`); + } + if (expectedEmission === "emitted") { + const documentedPaths = new Set(extractSourceLocationPaths(row.source)); + for (const path of usageByCode.get(code)) { + if (!documentedPaths.has(path)) { + errors.push(`Code '${code}' is emitted from '${path}', but docs/ERRORS.md does not list that source location.`); + } + } + for (const path of documentedPaths) { + if (!usageByCode.get(code).has(path)) { + errors.push(`Code '${code}' lists '${path}', but no semantic finding(ERROR_CODES., ...) call site exists there.`); + } + } + } + } + + for (const code of rowsByCode.keys()) { + if (!registryByCode.has(code)) { + errors.push(`docs/ERRORS.md lists '${code}', but it is absent from the ERROR_CODES registry.`); + } + } + + return { + errors, + registrySize: registryByCode.size, + emittedCount: [...usageByCode.values()].filter((paths) => paths.size > 0).length, + }; +} + +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + const result = await checkErrorReference(); + if (result.errors.length > 0) { + process.stderr.write(`Error reference check failed:\n${result.errors.map((message) => `- ${message}`).join("\n")}\n`); + process.exit(1); + } + process.stdout.write(`error reference: pass (${result.registrySize} registry entries, ${result.emittedCount} emitted)\n`); +} diff --git a/test/check-errors.test.ts b/test/check-errors.test.ts new file mode 100644 index 0000000..cbc9567 --- /dev/null +++ b/test/check-errors.test.ts @@ -0,0 +1,29 @@ +import { describe, expect, it } from "vitest"; +import { extractSourceLocationPaths, findFindingCodes } from "../scripts/check-errors.mjs"; + +describe("error reference call-site detection", () => { + it("only counts the first argument of a semantic finding call", () => { + const source = ` + const unrelated = ERROR_CODES.invalid_type; + const documentation = "finding(ERROR_CODES.missing_field, path, message)"; + // finding(ERROR_CODES.unknown_column, path, message) + /* finding(ERROR_CODES.non_numeric, path, message) */ + const object = { finding: () => undefined }; + object.finding(ERROR_CODES.invalid_enum, path, message); + finding( + ERROR_CODES.invalid_receipt, + path, + message, + ); + `; + + expect(findFindingCodes(source)).toEqual(["invalid_receipt"]); + }); + + it("supports documented source paths without accepting arbitrary path text", () => { + expect(extractSourceLocationPaths("`src/core/errors.ts`, `src/dataset/index.ts`")) + .toEqual(["src/core/errors.ts", "src/dataset/index.ts"]); + expect(extractSourceLocationPaths("registry only; no source path")) + .toEqual([]); + }); +});