Oserhiienko/knowledge base

.claude/skills/access-data/SKILL.md

@@ -0,0 +1,120 @@
+---
+name: access-data
+version: 0.2.1
+description: |
+  Pull rows from an external SQL database into a Datagrok `DataFrame` —
+  declare a connection, store credentials, author a parameterized query,
+  and call it from TypeScript. For plugin authors whose code needs data
+  from outside the browser. Produces `connections/<name>.json` +
+  `queries/<name>.sql` (registering a `<Package>:<Query>` function) and
+  the TS that resolves the query into a `DataFrame`.
+  Use when asked to "pull rows from a database into a dataframe", "wire
+  a parameterized SQL query into my plugin", or "store DB credentials
+  for a Datagrok connection". For REST endpoints see
+  `fetch-external-rest`; for bundled CSVs see `load-bundled-csv`; for
+  user/server files see `read-server-file`.
+triggers:
+  - pull rows from postgres into dataframe
+  - wire a parameterized sql query
+  - register a sql query as a package function
+  - store database credentials securely
+  - declare a datagrok data connection
+allowed-tools:
+  - Read
+  - Edit
+  - Write
+  - Bash
+---
+
+## Cited facts
+
+See [`facts.yaml`](./facts.yaml) — concrete API references for the `DG-FACT-NNN` citations used below.
+
+# access-data
+
+## When to use
+
+Your package needs rows from an external SQL database materialized as a
+`DataFrame` — with `grok publish` wiring the connection, credentials
+kept out of git, and one JS call to run the query. The spine is four
+steps: declare the connection, transfer credentials, author the query,
+call it from TypeScript.
+
+## Steps
+
+1. **Declare the data connection in `connections/<name>.json`.**
+   Scaffold with `grok add connection <name>`. Secrets go under
+   `credentials.parameters`, never inside `parameters` or `connString`.
+   See `DG-FACT-400` (file shape), `DG-FACT-401` (exact `dataSource`
+   strings — `Postgres`/`PostgresDart`/`MariaDB`/`MS SQL`/...; lowercase
+   forms don't match), `DG-FACT-409` (credentials routing).
+   ```json
+   {
+     "#type": "DataConnection", "name": "Chembl",
+     "parameters": {"server": "db.datagrok.ai", "port": 54325,
+                    "db": "chembl", "cacheResults": true},
+     "credentials": {"parameters": {"login": "", "password": ""}},
+     "dataSource": "Postgres"
+   }
+   ```
+   Expected: `grok publish` registers the connection as
+   `<Package>:<Name>` (case-insensitive at lookup).
+
+2. **Transfer credentials after the first deploy.** POST them once
+   published so they route through Datagrok's credentials store. The
+   dotted path targets one connection; `.../for/<Package>` (no dot)
+   targets the package itself (see `DG-FACT-409`).
+   ```bash
+   curl -X POST "$GROK_HOST/api/credentials/for/$PACKAGE.$CONNECTION" \
+     -H "Authorization: $API_KEY" -H "Content-Type: application/json" \
+     -d '{"login":"abc","password":"123"}'
+   ```
+   Expected: `200 OK`. Read back via `await _package.getCredentials()`
+   → `c.parameters['<key>']` (NOT `c.openParameters`).
+
+3. **Author the parameterized SQL query in `queries/<basename>.sql`.**
+   Scaffold with `grok add query <name>`. See `DG-FACT-402` (header
+   grammar: `--name`, `--friendlyName`, `--description`, `--connection`,
+   `--input`; `@name` references; `--end` separator) and `DG-FACT-403`
+   (one query per file when attaching sibling `<basename>.js` post-
+   process or `<basename>.layout` — binding is by basename).
+   ```sql
+   --name: ProteinClassification
+   --connection: Chembl
+   --input: int level = 1
+   select * from protein_classification where class_level = @level
+   ```
+   Expected: `grok publish` registers `<Package>:ProteinClassification`.
+
+4. **Execute the query from TypeScript.** Pass only `(queryName,
+   parameters)` — see `DG-FACT-404` for signature (3rd `adHoc` is
+   `@deprecated`; no 4th argument exists).
+   ```typescript
+   import * as grok from 'datagrok-api/grok';
+
+   const df = await grok.data.query(
+     `${_package.name}:ProteinClassification`, {level: 1});
+   grok.shell.addTableView(df);
+   ```
+   Expected: a `TableView` opens with the query result. For a scalar
+   return type use `grok.data.query<number>(...)`.
+
+## Common failure modes
+
+- 3rd/4th arg to `grok.data.query(...)` — drop them (`DG-FACT-404`).
+- `dataSource` set to lowercase `postgres`/`mssql` — exact match required (`DG-FACT-401`).
+- Credentials in `parameters`/`connString` — move under `credentials.parameters` and POST per step 2 (`DG-FACT-400`, `DG-FACT-409`).
+- Sibling `<basename>.js`/`.layout` not applied — split multi-query files (`DG-FACT-403`).
+- Sharing connection didn't share queries — sharing is one-way query→connection (`DG-FACT-410`).
+
+## See also
+
+- Source: `help/develop/how-to/db/access-data.md`
+  (mirror: `docs/_internal/articles-mirror/how-to/db/access-data.md`).
+- Knowledge (`docs/_internal/knowledge/knowledge-graph.md`):
+  `DG-FACT-400` – `DG-FACT-404`, `DG-FACT-409`, `DG-FACT-410`.
+- Related skills: `create-package` (scaffolds `connections/` and
+  `queries/`), `manage-credentials` (general form of step 2),
+  `fetch-external-rest` (REST via `grok.dapi.fetchProxy`),
+  `load-bundled-csv` (`_package.webRoot` + `grok.data.loadTable`),
+  `read-server-file` (`OpenServerFile` / `grok.dapi.files.*`).

.claude/skills/access-data/facts.yaml

@@ -0,0 +1,137 @@
+# Facts cited by the `access-data` skill.
+# Generated from knowledge-graph.md — do not edit by hand.
+# Total facts: 7
+
+- id: DG-FACT-400
+  kind: naming
+  scope: access-data
+  value: |
+    Package data connection = a JSON file under `packages/<Pkg>/connections/`.
+    REQUIRED fields: `parameters` (data-source-specific object) and
+    `dataSource` (connector name string). OPTIONAL: `name` (defaults to the
+    filename minus `.json`), `description`, `tags` (string[]),
+    `credentials.parameters` (login/password etc — routed through the
+    credentials store, NOT the world-readable `parameters` block).
+    Connection names are case-insensitive at lookup time (server behavior;
+    not observable in JS-API). Reachable as `<Pkg>:<ConnectionName>` after
+    deploy. `grok add connection <name>` scaffolds a template.
+  derives_from:
+    - article: help/develop/how-to/db/access-data.md:29-114
+    - code: packages/Chembl/connections/chembl.json
+    - code: packages/DBTests/connections/postgres-test-docker.json
+  contradicts: []
+  confidence: high
+
+- id: DG-FACT-401
+  kind: enum
+  scope: access-data
+  value: |
+    `dataSource` value is exact-string-matched server-side; observed
+    canonical values in production: `Postgres`, `PostgresDart`, `MariaDB`,
+    `MySQL`, `MS SQL` (with space), `Oracle`, `Snowflake`, `ClickHouse`,
+    `Neo4j`, `Access`, `Files`. The set is open (controlled by the
+    `connectors` registry on the server) but lowercase/abbreviated
+    spellings (`postgres`, `mssql`) DO NOT match. The article calls out
+    `Postgres` (line 38 example uses `PostgresDart`), `MariaDB` (line 106)
+    and `Files` (line 287). Each connector has its own required
+    `parameters` shape — see `help/access/databases/connectors/`.
+  derives_from:
+    - article: help/develop/how-to/db/access-data.md:38,106,287
+    - code: packages/DBTests/connections/*.json (10 distinct dataSource values)
+  contradicts: []
+  confidence: high
+
+- id: DG-FACT-402
+  kind: naming
+  scope: access-data
+  value: |
+    SQL queries live in `packages/<Pkg>/queries/`. A query file (`.sql`)
+    contains one or more queries, each opened by a header block of
+    `--`-prefixed annotations and terminated by `--end` (omit `--end` if
+    only one query per file). Recognized headers (per `--<key>: <value>`):
+    `--name` (required for multi-query files; otherwise defaults to
+    filename), `--friendlyName`, `--description`, `--connection: <Pkg>:<Conn>`
+    (omit if the package has only one connection), `--input: <type> <name>
+    [= <default>]` (one per parameter). Parameters in the SQL body are
+    referenced as `@name`. `grok add query <name>` scaffolds a template.
+  derives_from:
+    - article: help/develop/how-to/db/access-data.md:120-165
+    - code: packages/Chembl/queries/queries.sql:1-45
+  contradicts: []
+  confidence: high
+
+- id: DG-FACT-403
+  kind: naming
+  scope: access-data
+  value: |
+    Post-processing script + layout are linked to a query by EXACT basename
+    match: `queries/foo.sql` (the query), `queries/foo.js` (post-process
+    script — accepts the result DataFrame as input), `queries/foo.layout`
+    (layout applied to the result). Hard invariant: a file that needs an
+    associated `.js` or `.layout` MUST contain exactly ONE query — the
+    association is by filename, not by query name. Multi-query files break
+    the binding silently.
+  derives_from:
+    - article: help/develop/how-to/db/access-data.md:189-213
+  contradicts: []
+  confidence: high
+
+- id: DG-FACT-404
+  kind: namespace
+  scope: access-data
+  value: |
+    `grok.data.query<T = DataFrame>(queryName, queryParameters?: object | null
+    = null, adHoc?: boolean = false): Promise<T>` — async (`async` in source),
+    returns the resolved query result coerced to `T`. The 3rd `adHoc`
+    parameter is JSDoc-marked `@deprecated` and slated for removal — skill
+    code must omit it. `queryName` is `<Package>:<QueryName>` (the same
+    fully-qualified form the console uses). Internally calls
+    `grok_CallFunc(queryName, queryParameters, true, null)`. There is NO
+    4th polling-interval argument (signature has 3 positional slots).
+  derives_from:
+    - article: help/develop/how-to/db/access-data.md:179-185
+    - code: js-api/src/data.ts:255-262
+  contradicts: []
+  confidence: high
+
+- id: DG-FACT-409
+  kind: namespace
+  scope: access-data
+  value: |
+    Connection credentials are NEVER stored in the JSON committed to git —
+    after deploy, POST them to
+    `$(GROK_HOST)/api/credentials/for/$(PACKAGE_NAME).$(CONNECTION_NAME)`
+    with body `{"login":"...","password":"..."}` (any param keys per
+    connector) and headers `Authorization: $(API_KEY)`,
+    `Content-Type: application/json`. The API key is the user's personal
+    token from their profile page (e.g. https://public.datagrok.ai/u).
+    The dotted entity form targets a specific connection; omitting the
+    dot (`...PACKAGE_NAME` alone) targets the package itself for
+    package-level credentials. Read-back from JS:
+    `await _package.getCredentials(): Promise<Credentials | null>` →
+    `c.parameters['<key>']` (NOT `c.openParameters`, which is the redacted
+    display copy).
+  derives_from:
+    - article: help/develop/how-to/db/access-data.md:81-116
+    - code: js-api/src/entities/misc.ts:163-166
+    - code: js-api/src/entities/data-connection.ts:396-415
+  contradicts: []
+  confidence: high
+
+- id: DG-FACT-410
+  kind: naming
+  scope: access-data
+  value: |
+    Connection sharing is one-way: sharing a QUERY auto-shares its
+    underlying connection (including web/OpenAPI queries that ride on the
+    connection); sharing the CONNECTION does NOT auto-share its queries.
+    Inheritance rule per article — "access rights of a database connection
+    are inherited from the access rights of a query. However, the access
+    rights of a query don't inherit the access rights of the database
+    connection." Skill authors that distribute a package with private
+    queries must therefore share each query explicitly; sharing the
+    connection alone leaves queries inaccessible.
+  derives_from:
+    - article: help/develop/how-to/db/access-data.md:217-222
+  contradicts: []
+  confidence: medium