(Draft) NixOS vars by prescientmoon · Pull Request #547171 · NixOS/nixpkgs

prescientmoon · 2026-07-29T18:51:07Z

I've been working on a re-implementation of NixOS vars as part of the Google Summer of Code project, together with my mentor @lassulus. For info about the API, check the README included with this PR.

Things done

This PR implements a basic CLI for working with vars, a NixOS module that can be used together with the CLI, and some simple examples. There's also a simple NixOS test.

A bunch of concepts in the API have temporary names. Feel free to suggest better ones!

lassulus

all in all looks pretty good, some discussion around fixup we still have to clarify

also the documentation should probably live somewhere in the docs tree :)

nixos-discourse · 2026-08-07T18:40:03Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/modular-services-meeting-6-2026-08-07/79408/1

hsjobeki · 2026-08-14T07:42:38Z

There is now NixOS/nix#16300

Qubasa · 2026-09-14T12:19:46Z

Currently we cannot detect if a generator without dependents (a root generator) fails regenerating.

To solve this we could instead do:

Before generating, delete the metadata file, after it finished generating update the secret with a meta file containing a transaction ID.

if secret.meta is None then stale()

Dependent secrets can detect staleness because they track their parent transaction id in their meta file.

if parent.tid != secret.tid then stale()

prescientmoon · 2026-09-15T14:29:24Z

Currently we cannot detect if a generator without dependents (a root generator) fails regenerating.
...

@Qubasa

The transaction ID is pretty much equivalent to the ID we already store in the metadata. We already track parent staleness!

There's two reasons a root generator might regenerate:

The user passed --set or --generate, forcing it to regenerate
One (or more) of its files are missing (perhaps the user added a new file to the configuration!)

If we're in the latter case, then a failing generator will not break anything (the files are set after the generator runs). If we're in the former, then we'll keep regenerating as long as the user passes the same flags in future invocations. Whether we should keep regenerating even if the user removes those flags is questionable (I'd argue we shouldn't, hence why that's currently not the case).

Implementing the other behavior (remembering the failure even if the user stops passing the flags) wouldn't require any big change (we'd simply invalidate the metadata before running the generator).

I will say, this behavior isn't unique to root / dependency-less generators! Consider the following scenario: the user defines secrets A, B, and C. At first, B depends on A, and generation goes perfectly fine. Then, the user adds C as another dependency of B. The CLI detects the staleness of B (it's dependency list changed!), so an update is triggered. Say the update fails in this case, and the user once again removes C from B's dependencies. Since the dependency list matches the on-disk metadata, the CLI will now no longer attempt to regenerate B (even though it failed in the past!). This happens for the same reason the root-generator scenario you described happens, although I think it is what one would expect!

nixpkgs-ci

Thanks for contributing to the documentation.

Follow the documentation style guide, especially:

Show, don’t tell. Start with a minimal working example. Explain after the code.
Avoid meta-commentary. Don’t write "This section explains how to…".
Use present tense, imperative mood, and active voice. For example: "Run the command," not "The user should run the following command."
Avoid vague references like "this" when the referent is unclear. Write "This command creates a folder," not "This will create a folder."
Be direct. Avoid "should", "might", "typically", and "usually".
Cut filler words like "simply", "just", "easily", and "basically". Use "to" instead of "in order to".

For larger changes, such as adding or removing whole sections, ask the NixOS documentation team for review.

Samuel-Martineau · 2026-09-27T11:35:14Z

There is now NixOS/nix#16300

I am not sure whether this is actually related / relevant. It seems to be more about providing Nix with the credentials that it needs than general secret/variable management for NixOS systems...

PerchunPak

Nice talk! Sorry for leaving only nitpicks, don't really have time to review throughtly

Signed-off-by: prescientmoon <git@moonythm.dev>

…igs from Nix Signed-off-by: prescientmoon <git@moonythm.dev>

Signed-off-by: prescientmoon <git@moonythm.dev>

nixos-vars: nixos-vars: - fix naming inconsistencies (run & script, "backendModule" not being nixos-vars: clear enough (is it a generator or a prompt backend?) etc) nixos-vars: - make jsonify idempotent nixos-vars: - make jsonify accept both target and host nixpkgs instances, reusing nixos-vars: the one the given NixOS config has been built with in case none has been nixos-vars: given Signed-off-by: prescientmoon <git@moonythm.dev>

@PerchunPak

The entry was apparently in the toplevel gitignore. Thank you @PerchunPak!

…ITOR

…s gone)

con-f-use · 2026-10-02T15:07:33Z

Is there a timeline for getting this merged?

lassulus · 2026-10-04T09:16:58Z

Is there a timeline for getting this merged?

I would expect it after the next stable release or branch off if we don't find any major issues :)

DavHau

Thanks a lot for working on this! I finally got around having a look at it.

Stabilization Roadmap

I think this project is missing a plan on how it gets stabilized.

If I'm not mistaken this is currently in a POC state with nobody using it. There is no usable backend code and it is hidden behind a nixpkgs PR branch. Naturally, as there hasn't been any exposure to real use cases, there will be many issues with it and once this is in nixpkgs upstream, fixing the issues just creates unnecessary churn and maybe even prevent it from becoming as good as it could, since contributors might be reluctant on breaking backwards compat.

I would suggest to put this into a separate repo, add a usable backend, and let people use is as a dependency to their nix projects, then iterate and fix all the issues, and once it is mature enough, merge it into nixpkgs.

There have been so many surprises when we developed vars in clan-core, which only came up once the feature was used in real projects, and I think the same will happen to this implementation as well. Putting it into nixpkgs already will inevitably make a lot of people depend on it too soon, which will make it hard to change it in meaningful ways.

This is my most important point here. Everything else that follows is not that relevant until this is answered.

The rest of my review is mostly about high level issues. I haven't scrutinized every implementation detail, because I don't think this is the most useful thing to do here right now. I believe getting some exposure to real users would be much more effective in gaining confidence in this project than having a bunch of developers approving code that has never been used.

Testing and Type checking

The python code has no unit tests. Even if the code is not that large, this is a bad practice as the code will grow in the future and this situation invites people to add more untested code to it. Untested code is usually written in a hard-to-test style which is hard to fix later.

Some types are partially wrong. -> Setup a type checker that actually runs in CI.

optimally this code would survive a ruff and mypy check.
currently mypy reports 82 errors in 6 files

Unsafe example code

Large parts of the code added here seems to be unsafe example code that should not be used, yet it is exposed under a location under modulesPath that can easily be imported. I suggest moving all example code under nixos/tests/nixos-secrets and mention this in the docs. This would clearly underlines the intent that it should not be used.

Blind spots

This section is about known problems of the original vars implementation in clan-core, which are ignored by the implementation of this PR.

I'm aware that this PRs implementation intentionally wants to be lean and doesn't yet have the feature set that would trigger some of these issues, but still it is worthwhile mentioning them, because they will re-appear as soon as more features are added, if ignored by the fundamental architecture.

Performance Issues

In short, the way vars were implemented in clan-core introduced a significant performance issue, because deploying one machine required evaluating all generators of all other machines. Since generators can depend on any other nixos option or packages, the worst case of this scenario is having to evaluate all toplevel's of all machines whenever deploying any machine.

The source of all evil is a secret sharing feature that this PR's implementation doesn't have yet, but since this is a highly needed feature for any infrastructure with more than one machine it will inevitably be added sooner or later. Therefore this implementation should have at least a viable plan on how this can be added without re-introducing the performance bottleneck of clan vars.

Example of shared vars:
You have multiple CI builders that all need the same API key to push to some cache. The key is the same across all machines. You do not want to re-generate or re prompt it for each individual machine.

Example of shared vars with dependents:
You have multiple machines within the same VPN. Your VPN has a network ID which is shared across all machines. You only want to generate it once for all machines. If you ever change your network ID, all machines have to re-roll their keys (their generators depend on that shared network ID file).

The problem we have in clan with this is the following: Whenever a shared secret gets re-generated, we need to invalidate all dependant secrets of all machines, therefore we must know the graph of the dependents across all machines at once, which required us to evaluate all secrets across all machines on every single deployment.

Evaluation feedback cycles

In clan vars, we made it so that non-secret (eg. public) generated files became part of the repository and they could be read inside the nixos evaluation itself.

This allowed to create evaluation feedback cycles, where the properties or even existence of one generated file can be influenced by the existence of another generated file. This means a single evaluation of generators cannot guarantee you to know the whole final graph of of generators.

While this sounds like an edge case. It happened to use in practice many times over by accident to create these loops. Those usually surface as one of the following hard to debug defects:

generating vars crashes, because the evaluation of a generator depends on a another generated file that doesn't exist yet
generating all vars requires multiple (unknown amount of) rounds of vars evaluation and generation, because each generation leads to new vars being activated in the next evaluation.

This is pretty bad. We tried to prevent this by adding some mitigations, but those were quite expensive in terms of compute and potentially not reliable.

The best would be to explicitly discourage reading vars back into the nixos config. Backends should be discouraged to support reading back any generated state during evaluation time, to prevent these loops from happening.

Some minor notes I took while reading

what is store in secrets.store.<secret> = (unclear at the point in the docs where it first appears)
spelling and grammar mistakes (maybe pipe everything through an LLM to fix those)
config is written instead of config.nix in a bunch of examples
--set user=manual: why is manual a file or directory, why not prompt the user?

DavHau · 2026-10-21T05:05:37Z

+    #
+    # We're only doing it this way in order to keep the example portable
+    # (absolute paths would break it on other people's machines!).
+    settings.store.age.identity.host = toString ../common/key-host.txt;


Paths to the local filesystem should not be put into nix options. This becomes important as soon as the repo is used by more than one admin. Different people will use different keys and store them under different locations.

As a rule of thumb, any file path that is not committed to the repo should be set via an environment variable instead of a nix option. Changing this would also make your do *not* do this disclaimer unnecessary.

Also, storing important key material in plain text should be discouraged. Better to use ssh public keys in the example. Age supports them natively and they can be password protected.

Yeah, that's kind of the reason all the comments are there :p I will maybe try to make the example use env vars just to show people how to do it, I guess...

Now that I think about it, the disclaimer would still have to be there for the target machine's key...

DavHau · 2026-10-21T05:08:09Z

+    backends.defaults.prompt = "simple";
+
+    settings.store.age.publicKeys = [
+      "age13ar5t7vvsssmckjhjtngy3p5y0v4k896ecjrxveql9ysu8gxhe9sdar3k3" # Host


Best to use ssh public keys here instead. Most people already have ssh keys, age supports them natively and ssh key material is password protected most often. This is much better than storing raw key material on disk which is what the native age key generator does by default.

I agree in principle, although this is an example backend that is explicitly asking not to be used in production (and is mostly there to showcase the API), so I'm not sure it particularly matters. I'll consider switching it over to using SSH keys though.

DavHau · 2026-10-21T06:10:47Z

+          type = enum [
+            "hidden"
+            "line"
+            "multiline"


multiline-hidden is missing here

Hmm, how would a backend handle this in practice? The idea with multiline was that a backend would, for example, open the file in $EDITOR.

Still, if multiline-hidden is something people commonly need, I'd be open to adding it.

openssh private keys are encoded into multiple lines for example. So if you'd want to paste one for whatever reason, it would need that input mode. clan-core implements that by having the user paste into a blind (non echoing) terminal, then finish with Ctrl+D. The $EDITOR should not be used for secrets as it might cache it or heep a history of some sorts.

Hmm, I see the point regarding $EDITOR. I think sops-nix and the like do use $EDITOR for this kind of stuff though. Is that just a bad practice that's already common place?

DavHau · 2026-10-21T06:55:28Z

Oops, closed this by accident, then re-opened

prescientmoon · 2026-10-22T15:41:05Z

Hi @DavHau, thank you for the review!

Some unordered notes (not covering every point):

--set is meant to be file based. The plan was to also add an --edit flag which works the way you described it (by using $EDITOR), but I haven't gotten to it yet.
Re: nobody using this in practice. Yeah, the plan is for me and lassulus to switch our machines to it before merging. Even when merged, the module and whatnot wouldn't be enabled by default, so it would be an opt-in thingy for people who want to try it out. I don't have very strong thoughts in any direction here, as this has mostly been lassulus' plan.
I will try to take care of the typechecking issues.
The example code used to not live in modulePath, but it ended up being put there as a hack in order to allow the nixos tests to also access the files. I don't really remember the details to be honest, but I'll see whether they can be moved into the testing directory.
"secret sharing" is meant to be handled by the backend, I think? Let's take the VPN example. One would write a configuration file containing the secret logic, which would then get imported by all the machines. The first time the CLI is run for one of the machines, the network ID will get generated. After that, the backend will return the existing ID when other machines use the CLI. at I mean by "multi machine aware" is the fact the backend would need to do some sort of reference counting as to not delete files when one of the machines no longer needs them.
The feedback cycle issue is one we have discussed many, many (perhaps too many) times in the meetings, funnily enough :) The conclusion we have reached is indeed that backends should preferably just not allow the configuration to read the public files. I don't think it should be our job to handle what happens if a user decides to do so though. This is not a vars specific issue, after all! Indeed, one can write a NixOS configuration that contains a systemd oneshot service that edits the contents of /etc/nixos/configuration.nix, thus leading to similar cycles. Still, I will try to mention this explicitly in the documentation.
I'm not sure what store is either :p I think someone suggested this naming in one of the meetings, and it ended up sticking.

DavHau · 2026-10-26T06:02:33Z

the plan is for me and lassulus to switch our machines to it before merging

That's a good start but a larger user base would be much better.

Even when merged [...] it would be an opt-in thingy for people who want to try it out

Nix flakes feature was added with that same idea in mind. It became hard to change very quickly, despite its experimental status. Statuses don't mean anything. Exposure is what locks you in. Merging this into nixpkgs is the wrong kind of exposure for the state the project is in, I believe, and probably still will be after being tested by just 2 people IMO.

"secret sharing" is meant to be handled by the backend, I think?

I don't see how this solves the issue I was describing. You still need means of declaring something as shared in the nix interface. For example, you might have all your machines have the same root password or distinct ones. In both cases you'd want to simply have a module to import into all machines. Therefore shared must be an eval time feature and that's where the root of the problem lies. Whenever you change something that is shared there are potential side effects on other files of other machines that you cannot see or plan without evaluating all of them.

I'm almost certain that you will encounter some unforeseen problems once you actually implement this. One more reason to have a fully working backend and a separate repo to iterate on.

The conclusion we have reached is indeed that backends should preferably just not allow the configuration to read the public files

nice 👍

GetPsyched · 2026-10-26T07:03:11Z

As someone who's been waiting almost 2 years for this feature, I'm ready to consume this however; from nixpkgs (after merge), from this PR directly, or from a different repository this gets moved to for testing.

I agree with @DavHau that lock-in from exposure could block any refinements needed here. That said, the same thing was done in #370444 where development moved to Lassulus/vars. I hope the work doesn't fizzle out once again.

This will let the argument be extensible in the future.

I thought I had gotten a warning about the name changing, but apparently not!

…onfig.nix"

prescientmoon · 2026-10-29T14:44:57Z

@DavHau Re: sharing

Can you elaborate a bit more on "Whenever you change something that is shared there are potential side effects on other files of other machines that you cannot see or plan without evaluating all of them."?

It is true that changing something in a "shared" secret's declaration can lead to the configurations of other machines potentially changing as well. Nonetheless, such a change is meaningless unless the new versions of the other configurations are also to be deployed. But deploying said configurations would require evaluating them anyway, so I'm not sure where the issue lies... Is the issue that there's no way to do it incrementally?

hsjobeki · 2026-11-01T22:20:15Z

It is true that changing something in a "shared" secret's declaration can lead to the configurations of other machines potentially changing as well.

This is exactly the problem. You have to evaluate all machines, which can take several minutes. Depending how many machines you have, cost scaling linearly if you have more nixos configurations in lets say a flake.

I'd like to see examples how setups with more than one machine work.
Use cases that involve at least two machines are very common; Cross machine dependency edges are everywhere in the real world. In clan we also postponed it, then somebody proposed a shared: true workaround. Up to these days we where not able to resolve this. It costs users time, compute and causes weird bugs. Once this PR lands in nixpkgs refactoring will be much harder than in clan. That's why I advocate to run at least some small experiments to get some architecture right that scales beyond one machine, and can express dependency edges more freely without the mentioned linear performance penalty.

Is the issue that there's no way to do it incrementally?

The issue is performance, but also incremental deployments. If you have many machines you cannot expect all of them to evaluate or update at the same time.

I'd also like to use vars without NixOS. For devShells, containers, and so on. I think tight coupling to this beeing a NixOS module is a mistake.

prescientmoon marked this pull request as draft July 29, 2026 18:51

lassulus reviewed Aug 5, 2026

View reviewed changes

nixpkgs-ci Bot added the 8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` label Aug 5, 2026

KiaraGrouwstra reviewed Aug 7, 2026

View reviewed changes

Comment thread pkgs/by-name/ni/nixos-vars/src/nix_vars/eval.py Resolved

KiaraGrouwstra reviewed Aug 7, 2026

View reviewed changes

Comment thread pkgs/by-name/ni/nixos-secrets/src/nixos_secrets/config.py Resolved

mrVanDalo reviewed Aug 14, 2026

View reviewed changes

This comment was marked as outdated.

Sign in to view

lassulus reviewed Sep 11, 2026

View reviewed changes

Comment thread pkgs/by-name/ni/nixos-secrets/docs/03-store-backends.md Resolved

lassulus reviewed Sep 11, 2026

View reviewed changes

Comment thread pkgs/by-name/ni/nixos-secrets/docs/03-store-backends.md Resolved

nixpkgs-ci Bot reviewed Sep 22, 2026

View reviewed changes

prescientmoon force-pushed the master branch 2 times, most recently from b898670 to f6de203 Compare September 27, 2026 11:04

PerchunPak reviewed Sep 27, 2026

View reviewed changes

Comment thread nixos/modules/security/secrets/example/basic-config/default.nix Resolved

Comment thread pkgs/by-name/ni/nixos-secrets/src/nixos_secrets/args.py Resolved

Comment thread pkgs/by-name/ni/nixos-secrets/.gitignore Resolved

Comment thread pkgs/by-name/ni/nixos-secrets/file.txt Resolved

prescientmoon added 7 commits September 28, 2026 18:45

nixos-vars: Init

2439c9d

Signed-off-by: prescientmoon <git@moonythm.dev>

nixos-vars: Set up nix evaluation

a03772f

Signed-off-by: prescientmoon <git@moonythm.dev>

nixos-vars: Attempt to implement the proper logic for evaluating conf…

3c571ea

…igs from Nix Signed-off-by: prescientmoon <git@moonythm.dev>

nixos-vars: Add a comment explaining the stub target-host-nixpkgs import

b9cc660

Signed-off-by: prescientmoon <git@moonythm.dev>

nixos-vars: Delete some older code

daa8644

Signed-off-by: prescientmoon <git@moonythm.dev>

nixos-vars: Add a flake example

5007ad2

Signed-off-by: prescientmoon <git@moonythm.dev>

prescientmoon added 7 commits September 28, 2026 18:47

nixos-secrets: fix outdated doc script attribute naming

5c8e61a

nixos-secrets: fix unintentional lie about get being optional

eb67958

nixos-secrets: remove accidentally committed error message

6e70d0b

nixos-secrets: remove useless __pycache__

b9d40ac

The entry was apparently in the toplevel gitignore. Thank you @PerchunPak!

nixos-secrets: move all the scripts to pkgs.writeShellScript

5874d1f

nixos-secrets: remove references to deprecated typing thingies

cb7d110

nixos-secrets: fix tests

068e85d

prescientmoon force-pushed the master branch from a993a97 to 068e85d Compare September 28, 2026 16:47

prescientmoon added 2 commits September 28, 2026 19:33

nixos-secrets: fix the simple prompt backend not having access to $ED…

5ddcee4

…ITOR

nixos-secrets: garbage collect metadata files (when everything else i…

2539c73

…s gone)

nixpkgs-ci Bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 28, 2026

con-f-use mentioned this pull request Sep 28, 2026

WIP: implement a secret vars store in nixpkgs #370444

Closed

DavHau reviewed Oct 21, 2026

View reviewed changes

DavHau closed this Oct 21, 2026

DavHau reopened this Oct 21, 2026

prescientmoon added 6 commits October 29, 2026 15:02

nixos-secrets: Fix ruff linting errors

1bdb92c

nixos-secrets: Fix mypy errors & add checks to checkPhase

f9b8e16

nixos-secrets: make scripts take an object instead of a single arg

f725800

This will let the argument be extensible in the future.

nixos-secrets: fix silly option naming mistake

1de9b0d

I thought I had gotten a warning about the name changing, but apparently not!

nixos-secrets: fix "config" appearing in some doc paths instead of "c…

09abedb

…onfig.nix"

nixos-secrets: warn about evaluation cycles

b4b6d00

nixos-secrets: fix formatting

874975f

Conversation

prescientmoon commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Things done

Uh oh!

lassulus left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nixos-discourse commented Aug 7, 2026

Uh oh!

Uh oh!

Uh oh!

hsjobeki commented Aug 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Qubasa commented Sep 14, 2026

Uh oh!

prescientmoon commented Sep 15, 2026

Uh oh!

nixpkgs-ci Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Samuel-Martineau commented Sep 27, 2026

Uh oh!

PerchunPak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

con-f-use commented Oct 2, 2026

Uh oh!

lassulus commented Oct 4, 2026

Uh oh!

DavHau left a comment

Choose a reason for hiding this comment

Stabilization Roadmap

Testing and Type checking

Unsafe example code

Blind spots

Performance Issues

Evaluation feedback cycles

Some minor notes I took while reading

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

prescientmoon commented Jul 29, 2026 •

edited

Loading

lassulus left a comment •

edited

Loading

nixpkgs-ci Bot left a comment •

edited

Loading

DavHau commented Oct 26, 2026 •

edited

Loading

hsjobeki commented Nov 1, 2026 •

edited

Loading