Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions docs/developer-guide/working-with-ai.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
import {DeveloperDocsTabs} from '@site/src/components/docs/developer-docs-tabs';

# Working with AI Coding Agents

<DeveloperDocsTabs active="ai" />

AI coding agents can help design, implement, and debug luma.gl applications when they
work from the version actually installed in the application and can observe the result
in a real browser. This page describes a practical workflow for application developers
first, followed by additional guidance for contributors to the luma.gl repository.

## Start from local truth

Do not ask an agent to rely on its memory of luma.gl APIs. First give it the application
and ask it to identify the installed package versions:

```bash
npm ls @luma.gl/core @luma.gl/engine @luma.gl/shadertools @luma.gl/webgpu @luma.gl/webgl
```

Use the equivalent command for the project's package manager. The agent should then
inspect each installed package's `package.json`, exported TypeScript declarations, and
the documentation for that release. Installed declarations are the authority for exact
constructor props, methods, and types; current website documentation can describe a
newer release.

When a project is pinned to an older release, use the documentation in the matching
GitHub release branch. Ask the agent to cite the declaration or documentation page it
used when an API choice is uncertain.

## Choose the right luma.gl level

Give the agent the smallest API surface appropriate to the task:

| Level | Start here when | Typical packages |
| --- | --- | --- |
| Engine API | Building a rendered application, model, animation loop, geometry, picking, or common GPU transform | `@luma.gl/engine` plus one or more adapters |
| Core GPU API | Managing devices, buffers, textures, pipelines, bindings, render passes, or compute passes directly | `@luma.gl/core`, `@luma.gl/webgpu`, `@luma.gl/webgl` |
| Shader API | Assembling reusable shader modules, hooks, plugins, or matching WGSL and GLSL implementations | `@luma.gl/shadertools`, usually with Engine or Core |

Most applications should start with `Model` and `AnimationLoop` from Engine, then use
Core resources only where explicit control is useful. Shader tooling composes shader
code; it does not replace Engine or Core execution. See
[A Tale of Three APIs](/docs/api-guide) for the complete object model.

## Install the luma.gl skill

The repository contains an official progressive Agent Skill that routes application
design, portability, GPU debugging, and repository contribution tasks:

```bash
npx skills add visgl/luma.gl --skill lumagl
```

After installation, ask your agent to use the `lumagl` skill. A useful prompt names the
goal, supported backends, installed luma.gl version, and observable success condition:

> Use the lumagl skill to add this render path for WebGPU and WebGL 2. Confirm the
> installed package APIs, run the typecheck, exercise both backends in a real browser,
> and report console errors and screenshots.

The skill provides procedural judgment and debugging order. It does not copy the entire
API reference into every conversation.

## Give agents exact documentation

The website publishes [llms.txt](https://luma.gl/llms.txt), an index of current
tutorials, API guides, API references, and developer guides. Each listed page has a raw
Markdown sibling. For example:

- [Getting Started Markdown](https://luma.gl/docs/getting-started.md)
- [Portable Shaders Markdown](https://luma.gl/docs/api-guide/shaders/writing-portable-shaders.md)
- [Model API Markdown](https://luma.gl/docs/api-reference/engine/model.md)

Ask the agent to fetch only the pages needed for the task. The Markdown is generated
from the rendered site, so tabs, cards, and other MDX components become readable
content rather than JSX. Generated TypeDoc pages are included as raw Markdown
references as well.

`llms.txt` is an inference-time documentation index. It helps an agent select context;
it is not crawler access control, a training opt-out, or a substitute for accurate,
versioned documentation. Crawler policy belongs in mechanisms such as `robots.txt` and
provider controls.

## Require an observable verification loop

Typechecking is necessary but cannot prove that GPU code renders correctly. Ask the
agent to produce evidence in this order:

1. Run the application typecheck and relevant unit tests.
2. Open the actual application in a current browser, not a DOM-only test environment.
3. Capture console messages, page errors, failed requests, and shader compiler output.
4. Confirm the selected device and canvas context, then inspect layouts, bindings,
uploaded data, render-pass state, and draw counts.
5. Capture a screenshot after the expected frame has rendered.
6. Exercise WebGPU and WebGL 2 separately when the feature is intended to be portable.
7. Use browser GPU diagnostics or a frame debugger when logs and screenshots do not
isolate the fault.

For a blank canvas, debug the pipeline from the outside inward:
device and adapter availability → canvas context → shader compilation → layouts and
bindings → uploaded data → render pass and draw call. Changing shaders at random before
confirming device and binding state usually hides the original failure.

When a feature is genuinely WebGPU-only, state that constraint and test the unsupported
path explicitly. Do not ask the agent to invent a WebGL fallback for compute shaders,
storage textures, or another capability the backend does not provide.

## Working inside the luma.gl repository

Repository contributors should direct the agent to read the root `AGENTS.md` before
editing. It records coding conventions, ownership boundaries, and the stable command
surface. Prefer these root commands over runner-specific invocations:

```bash
yarn test-node
yarn test-browser
yarn test-headless
yarn test-coverage
yarn website-debug --example hello-triangle --backend webgpu-core
yarn website-debug --example hello-triangle --backend webgl2
```

`website-debug` records the final URL, a screenshot, a WebGPU capability probe, and page
diagnostics under `.playwright-artifacts/`. Ask the agent to inspect those artifacts and
report concrete evidence instead of concluding that a browser task passed because the
process exited successfully.

Before merge, follow the checks required by `AGENTS.md`. In particular, targeted tests
do not replace the final repository `yarn build` and `yarn test` gates. This workflow
does not add an AI disclosure requirement or otherwise change the contribution policy.

## How other frameworks inform this model

| Project | Forward-looking support | Lesson used by luma.gl |
| --- | --- | --- |
| [Next.js](https://nextjs.org/blog/agentic-future) | Agent-oriented documentation and upgrade workflows | Make version-aware framework knowledge easy to retrieve |
| [Nuxt](https://nuxt.com/docs/4.x/guide/ai/llms-txt) | Documented `llms.txt` variants for agent context | Publish a curated index and page-level Markdown |
| [Svelte](https://svelte.dev/llms.txt) | A public machine-readable documentation index | Give agents stable, direct documentation URLs |
| [TanStack](https://tanstack.com/intent/latest/docs/overview) | Intent and procedural guidance beyond API lookup | Encode task routing and judgment in a skill |
| [MapLibre](https://github.com/maplibre/maplibre-agent-skills) | Installable framework-specific Agent Skills | Ship the skill with the framework source |

luma.gl combines these ideas into four layers: accurate human documentation, raw
Markdown plus `llms.txt` for knowledge retrieval, one installable skill for procedural
work, and an offline corpus for manually comparing agent behavior. The repository
documents the [evaluation protocol](https://github.com/visgl/luma.gl/blob/master/test/llm/README.md);
CI validates the corpus but does not invoke a model. An embedded assistant, MCP server,
additional specialized skills, and a monolithic `llms-full.txt` are intentionally
deferred until evaluation or runtime-observability evidence shows they are needed.
1 change: 1 addition & 0 deletions docs/table-of-contents.json
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,7 @@
"label": "Developer Guide",
"items": [
"developer-guide/README",
"developer-guide/working-with-ai",
"developer-guide/installing",
"developer-guide/editing",
"developer-guide/debugging",
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,8 @@
"h3-js": "^4.4.0",
"pre-commit": "^1.2.2",
"vite": "^8.0.0",
"vitest": "^4.0.18"
"vitest": "^4.0.18",
"yaml": "^2.8.1"
},
"resolutions": {
"@docusaurus/plugin-client-redirects": "^3.9.2",
Expand Down
63 changes: 63 additions & 0 deletions skills/lumagl/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
name: lumagl
description: Design, implement, update, and debug luma.gl applications and repository changes with version-aware API selection, WebGPU/WebGL portability, browser-based GPU diagnosis, and project-specific verification. Use when working with @luma.gl packages, luma.gl shaders or resources, blank or incorrect canvases, backend compatibility, or the visgl/luma.gl repository.
---

# luma.gl

Work from the installed packages and observable browser behavior. Do not substitute
model memory for the consuming project's declarations, and do not claim rendering
success from typechecking alone.

## Establish local truth

1. Identify whether the task is in a consumer application or the luma.gl repository.
2. Inspect the package manager lockfile and installed `@luma.gl/*` versions.
3. Read the installed packages' `package.json` exports and TypeScript declarations for
every API used. Treat those declarations as authoritative.
4. Consult documentation for the same release. When using current
[luma.gl documentation](https://luma.gl/llms.txt), fetch only the raw Markdown pages
relevant to the task.
5. State any version or backend constraint that changes the implementation.

Never silently modernize code to an API that is absent from the installed declarations.
Do not infer API availability from a code sample without checking its version.

## Route the task

- For application architecture, package selection, or API-level decisions, read
[references/architecture.md](references/architecture.md).
- For code that must run on WebGPU and WebGL 2, or for a backend-specific feature, read
[references/portability.md](references/portability.md).
- For a blank canvas, shader failure, bad output, device loss, or binding problem, read
[references/debugging.md](references/debugging.md).
- For work inside `visgl/luma.gl`, read
[references/contributing.md](references/contributing.md) and the repository's root
`AGENTS.md` before editing.

Read every reference that applies; portability and debugging often overlap.

## Implement

1. Select Engine, Core GPU, and Shader APIs deliberately.
2. Preserve the application's existing package manager, build setup, style, and
resource-ownership conventions.
3. Keep the first change minimal and independently verifiable.
4. Label GPU resources with meaningful `id` values when the local API supports them.
5. Destroy owned GPU resources and animation infrastructure at the matching lifecycle
boundary. Do not destroy borrowed resources.
6. Keep backend-specific code behind an explicit capability or adapter boundary.

## Verify

Run the project's typecheck and focused tests, then exercise the actual application in
a current browser. Collect console output, page errors, failed requests, shader
compiler messages, and a screenshot after the expected frame.

For portable rendering, run WebGPU and WebGL 2 explicitly. For a WebGPU-only feature,
verify the supported path and a clear unsupported path. Report which backends and
browser environment were actually observed.

Do not stop at a successful process exit if the expected frame was not observed. When
the output is wrong, follow the debugging sequence in the reference instead of making
unrelated shader or pipeline changes.
74 changes: 74 additions & 0 deletions skills/lumagl/references/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Architecture and API Selection

## Source precedence

Use evidence in this order:

1. The consuming project's lockfile and installed package versions.
2. Installed `package.json` exports and TypeScript declarations.
3. Documentation from the matching release branch.
4. Current raw Markdown selected through `https://luma.gl/llms.txt`.
5. Model memory only for forming hypotheses to verify against the sources above.

A website version and an installed version may differ. Check all directly imported
`@luma.gl/*` packages because mixed package versions can create type or runtime
incompatibilities.

Useful inspection commands include:

```bash
npm ls @luma.gl/core @luma.gl/engine @luma.gl/shadertools @luma.gl/webgpu @luma.gl/webgl
yarn why @luma.gl/core
```

Follow the package's `types` or conditional `exports.types` entry to the declaration
file. Search declarations for exact props and overloads before writing a constructor or
method call.

## Select the API level

### Engine API

Start with `@luma.gl/engine` for models, geometry, animation, picking, scenegraph
helpers, dynamic resources, and common transforms. Most rendered applications should
begin here.

Use Engine when it can own the routine pipeline and redraw work. Do not drop to lower
levels merely because the underlying operation is GPU-backed.

### Core GPU API

Use `@luma.gl/core` when the application needs explicit devices, canvas contexts,
buffers, textures, samplers, shader layouts, bindings, pipelines, command encoders, or
render/compute passes. A concrete adapter from `@luma.gl/webgpu` or `@luma.gl/webgl`
must be available.

Core is portable at the luma.gl abstraction boundary, but individual capabilities are
not automatically portable. Check device features and limits before choosing a path.

### Shader API

Use `@luma.gl/shadertools` to assemble shader modules, plugins, hooks, defines, and
typed shader inputs. Shadertools composes shader source and contracts; Engine or Core
still creates resources and executes work.

## Design rules

- Prefer the highest level that exposes the control the task needs.
- Keep application-facing data and binding names stable across backends.
- Keep resource ownership explicit. Destroy objects created and owned by the feature;
do not destroy borrowed device, buffer, texture, or model resources.
- Prefer current `RenderPipeline` and resource APIs when declarations do not contain an
older `Program`-centric example.
- Request the narrowest device feature level and optional features that satisfy the
use case.
- If deck.gl already provides the required visualization abstraction, recommend it
rather than rebuilding layers, cameras, and interaction directly in luma.gl.

## Primary documentation

- `https://luma.gl/docs/api-guide.md`
- `https://luma.gl/docs/api-guide/gpu/gpu-initialization.md`
- `https://luma.gl/docs/api-guide/gpu/gpu-resources.md`
- `https://luma.gl/docs/api-guide/shaders/shader-assembly.md`
- `https://luma.gl/docs/api-reference.md`
66 changes: 66 additions & 0 deletions skills/lumagl/references/contributing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Contributing in visgl/luma.gl

## Read repository instructions

Read the root `AGENTS.md` before editing, then check for more specific instructions in
the affected tree. Repository instructions are authoritative for setup, style,
ownership boundaries, validation, and merge preparation.

Preserve unrelated working-tree changes. Follow established TypeScript and
documentation conventions, and keep new agent support on the existing contributor
command surface.

## Stable root commands

Use root scripts rather than reaching into test-runner internals:

```bash
yarn test-node
yarn test-browser
yarn test-headless
yarn test-coverage
yarn website-debug
```

`yarn test-node` is a focused check. It does not replace the repository-wide build or
combined test gate. Follow the final command sequence in `AGENTS.md`, including
formatting after changes and the required `yarn build` and `yarn test`.

Reusable Vitest and Playwright wiring lives in
`dev-modules/devtools-extensions/`. Repository-specific overrides live in
`.ocularrc.js`. Change the reusable workspace for shared runner behavior and the root
configuration for luma.gl-specific policy.

## Browser diagnosis

Run an example with an explicit backend:

```bash
yarn website-debug --example hello-triangle --backend webgpu-core
yarn website-debug --example hello-triangle --backend webgl2
```

The runner writes:

- `.playwright-artifacts/website-playwright.png`;
- `.playwright-artifacts/webgpu-probe.json`;
- `.playwright-artifacts/page-diagnostics.json`;
- `.playwright-artifacts/last-url.txt`.

Inspect the artifact contents. A zero exit code is not evidence that the expected
pixels rendered or that the intended backend was selected.

## Scope and policy

Use the same contribution standards for human- and agent-authored changes. This skill
does not create disclosure requirements or other contribution policy. It only makes
the existing instructions, verification loops, and architecture boundaries easier for
an agent to follow.

## Repository sources

- `AGENTS.md`
- `dev-modules/devtools-extensions/docs/llm-friendly-test-setup.md`
- `dev-modules/devtools-extensions/docs/vitest.md`
- `dev-modules/devtools-extensions/docs/playwright.md`
- `.ocularrc.js`
Loading
Loading