Skip to content

feat: yaml config provider - #4346

Open
zilto wants to merge 2 commits into
develfrom
feat/yaml-config-provider
Open

feat: yaml config provider#4346
zilto wants to merge 2 commits into
develfrom
feat/yaml-config-provider

Conversation

@zilto

@zilto zilto commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Solves #4345 (and #1596). It expands on previous work that allowed custom config provider.

Changes

There are a lot of files and lines of code changes, but it's inflated by .yaml test cases.

The main changes:

  • move logic of the toml config provider to a base class SettingsDocProvider
  • keep read/write logic in toml and yaml specific classes

Future work

  • update the docs to make more mentions of config.yaml and secrets.yaml
  • ensure this is supported by dlthub
  • ensure the dltHub AI Harness handles .yaml files properly (e.g., MCP tool to read values, mentions in skills)

@zilto zilto closed this Aug 11, 2026
@zilto zilto reopened this Aug 11, 2026
@zilto zilto changed the title feat/yaml config provider feat: yaml config provider Aug 11, 2026
@zilto
zilto requested a review from rudolfix August 11, 2026 20:59
@zilto zilto self-assigned this Aug 11, 2026
@zilto
zilto requested a review from burnash August 14, 2026 17:25
@zilto zilto added enhancement New feature or request release-highlight Changes to highlight in release notes needs confirmation Needs confirmation that the change/PR is actually required. labels Aug 18, 2026

@rudolfix rudolfix left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things before full review:

  • I'd prune providers that get into initial provider list to declutter exception messages (see comments)
  • I'd whitelist yaml extensions in ConfigurationFileSelector to start sending this to runtime. enabling runtime to pass through yaml profiles is separate (right now yaml files will be pruned), but not hard
  • tools to redact and update config/secret files that agents are using will need to be rewritten. right now they handle toml only. also rewriting a yaml path will need some kind of convention. we'll need to maybe switch to yaml parser that preserves comments and whitespaces (we were using it in dlthub)
  • there are very long module and function docstrings. those should be split and converted into comments. right now they are too sloppy. you can ask claude to apply rules from @docstrings.md or or try /review-vocabulary (which is based on Boeing simple simple English) and see what happens

config_toml = ProfileConfigTomlProvider(self.settings_dir, profile_name, self.global_dir)
secrets_yaml = ProfileSecretsYamlProvider(self.settings_dir, profile_name, self.global_dir)
config_yaml = ProfileConfigYamlProvider(self.settings_dir, profile_name, self.global_dir)
warn_on_toml_yaml_collision(secrets_toml, secrets_yaml)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here I'd remove yaml providers that do not have any locations present from the list of providers below. otherwise they will be reported in configuration errors as empty providers without files. we can be smarter and also prune tomls if there are no locations defined (for both config and secrets)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request needs confirmation Needs confirmation that the change/PR is actually required. release-highlight Changes to highlight in release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants