Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/workflows/datasets.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ on:
# Every Monday at 09:00 America/Sao_Paulo (12:00 UTC), same day as Dependabot.
- cron: "0 12 * * 1"
workflow_dispatch:
inputs:
datasets:
description: "Datasets to rebuild, separated by spaces (e.g. `ncm cbo`). Empty rebuilds all of them."
required: false
default: ""
type: string

permissions:
contents: read
Expand All @@ -29,7 +35,11 @@ jobs:
uses: ./.github/actions/setup

- name: Rebuild datasets
run: npm run build:data
env:
DATASETS: ${{ inputs.datasets }}
run: |
read -ra datasets <<< "$DATASETS"
npm run build:data -- "${datasets[@]}"

- name: Validate the regenerated datasets
run: npm run check && npx vp test --run
Expand Down
42 changes: 21 additions & 21 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@
"overrides": {
"@puppeteer/browsers@2": "^3.2.2",
"basic-ftp@5": "^5.3.1",
"brace-expansion@2": "^2.1.2",
"brace-expansion@2": "^2.1.7",
"eslint-plugin-sonarjs": {
"typescript": "6.0.3",
"ts-api-utils": {
Expand Down
32 changes: 31 additions & 1 deletion scripts/data.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
#!/usr/bin/env node

/**
* Rebuilds the embedded datasets from their official sources, then lints and formats the
* generated files.
*
* Usage:
* node scripts/data.ts [dataset...]
* Rebuild every dataset, or only the named ones (`ncm`, `cbo`, ..., the generator file names
* without `.ts`). A generator that fails is named on stderr and makes the script exit with 1.
*/

import { spawn } from "node:child_process";
import { resolve } from "node:path";

Expand Down Expand Up @@ -67,10 +77,30 @@ const generatedFiles = [
"./src/is-valid-ncm/constants.ts",
];

const requested = process.argv.slice(2).map((name) => name.replace(/\.ts$/, ""));
const known = generators.map((generator) => generator.replace(/\.ts$/, ""));
const unknown = requested.filter((name) => !known.includes(name));

if (unknown.length > 0) {
console.error(`Unknown datasets: ${unknown.join(", ")}. Known: ${known.join(", ")}.`);
process.exit(1);
}

const selected =
requested.length === 0
? generators
: generators.filter((_, index) => requested.includes(known[index]));

const results = await Promise.all(
generators.map((generator) => run("node", [resolve(scriptsDirectory, generator)])),
selected.map((generator) => run("node", [resolve(scriptsDirectory, generator)])),
);

const failed = selected.filter((_, index) => results[index] !== 0);

if (failed.length > 0) {
console.error(`Failed to rebuild: ${failed.join(", ")}`);
}

// Lint and format before checking the generators, so a failing generator never leaves
// unformatted files behind in the working tree. `vp fmt` runs last because `vp lint --fix`
// rewrites code without reformatting it.
Expand Down
9 changes: 5 additions & 4 deletions src/is-valid-ncm/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
* NCM (Nomenclatura Comum do Mercosul) 8 digit codes in force on the generation date, in ascending
* order, packed as base 36 differences by `packCodes` (scripts/lookup-table.ts) (one per source
* line, joined by line continuations). A code is included when the generation date falls within its
* Siscomex `Data_Inicio`/`Data_Fim` range (both inclusive), which also keeps codes that are valid
* today but carry a scheduled future end date, not only the ones with no end date (`Data_Fim:
* "31/12/9999"`).
* Siscomex `Data_Inicio`/`Data_Fim` range (both inclusive), which also keeps codes that are
* valid today but carry a scheduled future end date, not only the ones with no end date
* (`Data_Fim: "31/12/9999"`).
*
* Generated by `node ./scripts/ncm.ts`. Do not edit by hand.
*
Expand Down Expand Up @@ -9213,7 +9213,8 @@ a,\
1y,\
jq,\
2ee,\
28,\
a,\
1y,\
s2,\
2s,\
jq,\
Expand Down
Loading