feat(schema): generate config JSON Schema from the provider registry by viniciusdc · Pull Request #507 · nebari-dev/nebari-infrastructure-core

viniciusdc · 2026-07-24T18:49:01Z

What

Generates JSON Schema documents for nebari-config.yaml and every registered provider's Config struct, straight from the Go types, and commits them under schemas/ for the docs site to consume.

This supersedes #362. That branch was 41 commits behind main and built entirely on the old pkg/provider/ (singular) layout, which the provider refactor removed in favour of pkg/providers/{cluster,dns}/. Rather than rebase two commits across a 41-commit layout change, this re-homes the pipeline onto current main and folds in the content fixes the docs rendering work surfaced.

Changes

ConfigType() reflect.Type on the cluster and DNS provider interfaces, with one-line implementations on all seven registered providers (aws, azure, gcp, hetzner, local, existing, cloudflare). Added as a small configtype.go per package to keep the diff local.
pkg/configschema wraps invopop/jsonschema: reads yaml tags, package-qualifies $defs keys (so aws.Config and longhorn.Config don't merge), emits additionalProperties: false, and pulls field descriptions from godoc via AddGoComments.
pkg/nic/config_types.go exposes RegisteredConfigTypes through the registry, so the generator enumerates providers with no hard-coded list - adding a provider extends the output automatically.
cmd/schemagen writes schemas/manifest.json ({providers, dns, top_level}), nebari-config.json, and providers/<name>.json. make schemas regenerates; .github/workflows/schemas.yml is a paths-filtered drift gate that fails if the committed output diverges from the code (kubebuilder CRD-regen pattern, no auto-commit).
schemas/ committed in-tree (6 cluster providers + Cloudflare + top-level + manifest).

Content fixes surfaced by the docs rendering

enum tags on certificate.type (selfsigned/letsencrypt/existing) and the AWS node-group taint.effect (NO_SCHEDULE/NO_EXECUTE/PREFER_NO_SCHEDULE), so the allowed values are structured rather than prose the reader has to parse.
dropped an internal issue link from the trust_bundle godoc that was leaking onto the generated, user-facing schema.

Docs consumption

The docs site renders these schemas live - see the config-schema reference PoC and its Netlify preview (nebari-docs PR: https://redirect.github.com/nebari-dev/nebari-docs/pull/710). It currently tracks this branch via raw.githubusercontent.com; once this lands on a tagged release the docs pin to the tag.

Test plan

go build ./... clean; go vet ./... clean; gofmt clean
go test ./... passes (the interface change compiles against all providers and test doubles)
make schemas is deterministic - regenerating on a clean tree yields no diff (the CI drift gate)
Generated schemas reflect current main (e.g. Cloudflare zone_name required; certificate/taint enums present)
go run ./cmd/schemagen -providers aws,hetzner narrows correctly

Deferred (follow-ups)

FormatYAML (the commented-YAML reference) is still stubbed.
Broader examples/default struct tags and provider godoc coverage - most provider fields still have no description; the schema surfaces them the moment the godoc lands.
Publishing schemas/ per release tag + a schema-versions.json so the docs version toggle can list real versions (currently the docs track this branch).

Re-home the config schema-generation pipeline onto the current provider layout (pkg/providers/{cluster,dns}), superseding the stale branch behind #362 which was 41 commits behind main and built on the removed pkg/provider/ layout. - Add ConfigType() reflect.Type to the cluster and DNS provider interfaces, with one-line implementations on all seven registered providers. - pkg/configschema wraps invopop/jsonschema: reads yaml tags, package- qualifies $defs keys so aws.Config and longhorn.Config don't collide, emits additionalProperties:false, and pulls field descriptions from godoc. - pkg/nic/config_types.go exposes RegisteredConfigTypes through the registry, so the generator enumerates providers with no hard-coded list. - cmd/schemagen writes schemas/manifest.json, nebari-config.json, and providers/<name>.json. `make schemas` regenerates; a paths-filtered CI workflow fails on drift. - schemas/ is committed in-tree for the docs site to consume. Content fixes surfaced while rendering these schemas in the docs: - enum tags on certificate.type and the AWS node-group taint effect, so the allowed values are structured instead of buried in prose. - drop an internal issue link from the trust_bundle godoc that was leaking onto the generated, user-facing schema. Refs #40, #360.

Add jsonschema default/enum tags for fields whose defaults are applied in code, so the config reference shows sane values instead of blanks: - aws load_balancer_scheme: enum internet-facing|internal, default internet-facing (LoadBalancerSchemeOrDefault). - aws load-balancer-controller chart_version: default 3.2.1 (defaultLBCChartVersion). - existing storage_class: default standard (defaultStorageClass). - azure node pool mode: enum System|User, default User. Values are taken from the defaulting code, not the example comments, so the documented default matches what the CLI actually applies. Regenerated schemas/ accordingly.

- cmd/schemagen: tighten generated-file permissions to satisfy gosec (G301 dir 0o750, G306 files 0o600). Git records mode 0644 for the committed schemas regardless, so the artifact is unaffected. - Bump the vulnerability gate's govulncheck to v1.6.0. v1.4.0 crashes ("ForEachElement called on type containing *types.TypeParam") on the generic types pulled in transitively by invopop/jsonschema, which the schema generator now depends on. v1.6.0 analyzes the module cleanly and the gate stays green (no reachable, fixable vulnerabilities). - Drop an unnecessary type parameter from sortedKeys now that its only caller passes map[string]reflect.Type.

…iles Reduce the footprint of the schema generator: - Remove ConfigType() from the cluster and DNS provider interfaces and the seven per-provider configtype.go files. schemagen now references the provider Config types directly (clusterConfigTypes / dnsConfigTypes maps), so the runtime interfaces carry no schema-only method. As an internal build tool a small hard-coded map is an acceptable trade; keep it in sync with pkg/nic/registry.go (CI drift check surfaces staleness). - Delete pkg/nic/config_types.go (RegisteredConfigTypes), now unused. - Mark the generated output: add schemas/README.md ("do not edit by hand") and a "_comment" field in manifest.json, since JSON has no comment syntax. The docs consumer ignores the extra field. Provider schemas are byte-identical; only manifest.json changes (the marker).

…bility Replace schemagen's hard-coded provider->type map (a second place to keep in sync) with a registry-driven lookup, without putting a schema-only method on the Provider interface: - Add an optional ConfigTyped capability interface to the cluster and dns packages (ConfigType() reflect.Type). It is deliberately not part of the Provider interface, mirroring the existing optional-interface pattern in pkg/nic, so out-of-tree providers (ADR-0004) and test fakes are unaffected. - Each in-tree provider implements ConfigType() inline in its provider.go, so the config type lives with the provider - the single source of truth - and is reusable later by `nic init`. - pkg/nic/config_types.go walks the registry and reads each provider's type through a ConfigTyped type assertion; cmd/schemagen consumes that instead of importing every provider package and listing them by hand. Generated schemas are byte-identical; this is purely a wiring change.

…yped RegisteredConfigTypes skips providers without a config type, so a provider added to the registry that forgets ConfigType() would be silently omitted from the generated schemas. Assert every registered cluster and DNS provider is covered, so the omission fails loudly in CI. Verified it fails when a provider's ConfigType method is removed.

viniciusdc · 2026-07-25T14:53:42Z

 // Name returns the provider name
+// ConfigType reports the Go type of this provider's configuration struct
+// (the optional cluster.ConfigTyped capability used by schema generation and
+// config scaffolding).
+func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }
+


Suggested change

// Name returns the provider name

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// Name returns the provider name

viniciusdc · 2026-07-25T14:54:18Z

 // Name returns the provider name used in cluster.azure: dispatch.
+// ConfigType reports the Go type of this provider's configuration struct
+// (the optional cluster.ConfigTyped capability used by schema generation and
+// config scaffolding).
+func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }
+


Suggested change

// Name returns the provider name used in cluster.azure: dispatch.

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// Name returns the provider name used in cluster.azure: dispatch.

viniciusdc · 2026-07-25T14:54:34Z

 // Name returns the provider name.
+// ConfigType reports the Go type of this provider's configuration struct
+// (the optional cluster.ConfigTyped capability used by schema generation and
+// config scaffolding).
+func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }
+


Suggested change

// Name returns the provider name.

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// Name returns the provider name.

viniciusdc · 2026-07-25T14:54:50Z

 // Name returns the provider name
+// ConfigType reports the Go type of this provider's configuration struct
+// (the optional cluster.ConfigTyped capability used by schema generation and
+// config scaffolding).
+func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }
+


Suggested change

// Name returns the provider name

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// Name returns the provider name

viniciusdc · 2026-07-25T14:55:14Z

+// (the optional cluster.ConfigTyped capability used by schema generation and
+// config scaffolding).
+func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }
+


Suggested change

// Name returns the provider name

viniciusdc · 2026-07-25T14:55:28Z

 // Name returns the provider name
+// ConfigType reports the Go type of this provider's configuration struct
+// (the optional cluster.ConfigTyped capability used by schema generation and
+// config scaffolding).
+func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }
+


Suggested change

// Name returns the provider name

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// ConfigType reports the Go type of this provider's configuration struct

// (the optional cluster.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// Name returns the provider name

viniciusdc · 2026-07-25T14:55:45Z

 // Name returns the provider name.
+// ConfigType reports the Go type of this provider's configuration struct
+// (the optional dns.ConfigTyped capability used by schema generation and
+// config scaffolding).
+func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }
+


Suggested change

// Name returns the provider name.

// ConfigType reports the Go type of this provider's configuration struct

// (the optional dns.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// ConfigType reports the Go type of this provider's configuration struct

// (the optional dns.ConfigTyped capability used by schema generation and

// config scaffolding).

func (p *Provider) ConfigType() reflect.Type { return reflect.TypeFor[Config]() }

// Name returns the provider name.

dcmcand · 2026-08-10T10:36:45Z

@viniciusdc should this be closed in favor of #567 ?

viniciusdc · 2026-08-18T13:59:41Z

superseeded by #600

viniciusdc requested review from aktech, dcmcand, marcelovilla, oldsj and tylerpotts as code owners July 24, 2026 18:49

viniciusdc mentioned this pull request Jul 24, 2026

feat(schema): config schema generation pipeline (foundation) #362

Closed

7 tasks

viniciusdc added 2 commits July 24, 2026 16:34

viniciusdc mentioned this pull request Jul 24, 2026

Add spot-instance support to cluster provider node groups (Azure), unify naming (rename GCP Preemptible to Spot) #508

Open

6 tasks

viniciusdc added 3 commits July 24, 2026 19:45

viniciusdc commented Jul 25, 2026

View reviewed changes

This was referenced Aug 5, 2026

Epic: Distribute NIC via pixi/prefix.dev and ship Nebari deployments as Nebi workspaces #552

Open

Add a version-pinned $schema modeline to starter configs #562

Open

This was referenced Aug 13, 2026

Publish per-provider Nebi starter workspaces to quay.io/nebari #560

Closed

feat(config): reject CHANGEME placeholder values in validation #583

Merged

Update ADR-0005 with the config-surface and distribution decision #565

Open

viniciusdc force-pushed the feat/config-schema-gen-v2 branch from 4845ea2 to ef457d2 Compare August 15, 2026 17:25

viniciusdc changed the base branch from main to generate_docs August 15, 2026 17:25

viniciusdc mentioned this pull request Aug 15, 2026

feat(schema): generate config JSON Schema, folded into docgen #600

Open

6 tasks

viniciusdc changed the base branch from generate_docs to main August 15, 2026 17:51

viniciusdc force-pushed the feat/config-schema-gen-v2 branch from ef457d2 to 4845ea2 Compare August 15, 2026 17:51

viniciusdc closed this Aug 18, 2026

viniciusdc mentioned this pull request Aug 21, 2026

starters: generate and publish local + aws Nebi workspaces #594

Merged

Navigation Menu

Uh oh!

feat(schema): generate config JSON Schema from the provider registry - #507

feat(schema): generate config JSON Schema from the provider registry#507
viniciusdc wants to merge 6 commits into
mainfrom
feat/config-schema-gen-v2

viniciusdc commented Jul 24, 2026 •

edited

Loading

Uh oh!

viniciusdc Jul 25, 2026

Uh oh!

viniciusdc Jul 25, 2026

Uh oh!

viniciusdc Jul 25, 2026

Uh oh!

viniciusdc Jul 25, 2026

Uh oh!

viniciusdc Jul 25, 2026

Uh oh!

viniciusdc Jul 25, 2026

Uh oh!

viniciusdc Jul 25, 2026

Uh oh!

dcmcand commented Aug 10, 2026

Uh oh!

viniciusdc commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

viniciusdc commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Changes

Content fixes surfaced by the docs rendering

Docs consumption

Test plan

Deferred (follow-ups)

Uh oh!

viniciusdc Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

viniciusdc Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

viniciusdc Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

viniciusdc Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

viniciusdc Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

viniciusdc Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

viniciusdc Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

dcmcand commented Aug 10, 2026

Uh oh!

viniciusdc commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

viniciusdc commented Jul 24, 2026 •

edited

Loading