nix-configs

Repository containing my personal Nix (NixOS, Home Manager etc.) configurations.

Basic structure:

modules/ - all modules live here, they MUST be turned off by default (side-effect free imports),
- modules/nixos - NixOS specific modules
- modules/home-manager - Home Manager modules
- modules/nix-darwin - nix-darwin modules
- modules/shared - modules & configs shared by more than 1 "target"
  - modules/shared/universal - universal modules, often defining options to be handled elsewhere
packages/, some personal/in-house/out-of-band tools

Generally I aim to hide as much as possible behind *.enable options.

Overview

This is an incomplete list of incorporated software/systems:

disks: ZFS on LUKS through disko
desktop: Sway (Wayland), as much as possible through Home Manager
shell: Fish
users: Home Manager
MISSING: development environment

Notes

Tasks

Repository tasks are tracked as one file per task under docs/tasks/ — a finished task is tagged status: done in frontmatter and gains a sibling <task>.done.md solution file. See docs/tasks/README.md for the full convention. (The legacy root TASKS.md is being phased out.)

Custom ISO installer

see https://bmcgee.ie/posts/2022/12/setting-up-my-new-laptop-nix-style/

# building the installer from packages/install-iso
set INSTALL_ISO_PATH "$(nom build '.#install-iso' --no-link --print-out-paths --print-build-logs)/iso/kdn-nixos-install-iso.iso"
sudo dd bs=1M if="$INSTALL_ISO_PATH" of=/dev/disk/by-id/usb-SanDisk_Cruzer_Blade_02000515031521144721-0:0 status=progress
sudo dd bs=1M if="$INSTALL_ISO_PATH" of=/dev/disk/by-id/usb-_Patriot_Memory_070133F17AC22052-0:0 status=progress
sudo dd bs=1M if="$INSTALL_ISO_PATH" of=/dev/disk/by-id/usb-Samsung_Portable_SSD_T5_1234567D585A-0:0 status=progress
scp "$INSTALL_ISO_PATH" root@kvm-fa56.lan.etra.net.int.kdn.im:/data/nixos-amd64.iso
# boot the machine and ssh into it
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null kdn@kdn-nixos-install-iso

Golden path for bootstrapping new physical machine

build the install-iso and boot it
prepare a new config at host/${HOSTNAME}/default.nix
- put kdn.profile.machine.baseline.enable = true
- look through kdn.hardware.{gpu,cpu}.{intel,amd}
- set up zramSwap & boot.tmp.tmpfsSize
- figure out missing boot.initrd.{availableK,k}ernelModules
boot the install-iso

set up a disk configuration gathering data from the running system,

(
  # hashtag comment denotes defaults
  # /* */ are just comments
  let
    cfg = config.kdn.disks;
    d1 = "<DISK_1_NAME>-${config.networking.hostName}";
    d1Cfg = cfg.luks.volumes."${d1}";
    d2 = "<DISK_2_NAME>-${config.networking.hostName}";
    d2Cfg = cfg.luks.volumes."${d2}";
  in
  {
    kdn.disks.enable = true;
    #kdn.disks.zpool-main.name = "<HOSTNAME>-main";
    #kdn.disks.devices."${config.kdn.disks.defaults.bootDeviceName}" = { type = "gpt"; content.partitions.ESP = { size = "4096M"; ... }; };
    #kdn.disks.zpools."${cfg.zpool-main.name}" = { };
    #disko.devices.zpool."${cfg.zpool-main.name}".datasets = {
    # "${hostname}/nix-system/nix-store" = { mountpoint = "/nix/store"; ... };
    # "${hostname}/nix-system/nix-var" = { mountpoint = "/nix/var"; ... };
    #}

    /* point it at the right detached `/boot` disk (USB flash drive) */
    kdn.disks.devices."${config.kdn.disks.defaults.bootDeviceName}".path = "/dev/disk/by-id/usb-<CORRECT_IDENTIFIER>";
    #disko.devices.disk."${config.kdn.disks.defaults.bootDeviceName}" = { content.type = "gpt"; content.partitions = { ... }; ... };
    #kdn.disks.devices."${d1}" = { type = "luks"; path = d1Cfg.targetSpec.path; };
    #disko.devices.disk."${d1}" = { type = "luks"; name = "${d1}-crypted"; ... };
    #kdn.disks.devices."${d2}" = { type = "luks"; path = d2Cfg.targetSpec.path; };
    #disko.devices.disk."${d1}" = { type = "luks"; name = "${d1}-crypted"; ... };

    kdn.disks.luks.volumes."${d1}" = {
      #target.deviceKey = d1;
      targetSpec.path = "/dev/disk/by-id/<DISK_PATH>";
      uuid = "<uuidgen result>";
      #keyFile = "/tmp/${d1}.key";
      #header.deviceKey = config.kdn.disks.defaults.bootDeviceName;
      #header.partitionKey = d1;
      headerSpec.num = 2;
      #zpool.name = cfg.zpool-main.name;
    };

    kdn.disks.luks.volumes."${d2}" = {
      #target.deviceKey = d2;
      targetSpec.path = "/dev/disk/by-id/<DISK_PATH>";
      uuid = "<uuidgen result>";
      #keyFile = "/tmp/${d1}.key";
      #header.deviceKey = config.kdn.disks.defaults.bootDeviceName;
      #header.partitionKey = d2;
      headerSpec.num = 3;
      #zpool.name = cfg.zpool-main.name;
    };

    #kdn.disks.impermanence."sys/config".snapshots = true;
    #kdn.disks.impermanence."sys/cache".snapshots = false;
    #kdn.disks.impermanence."sys/data".snapshots = true;
    #kdn.disks.impermanence."sys/state".snapshots = false;
    #kdn.disks.impermanence."usr/config".snapshots = true;
    #kdn.disks.impermanence."usr/cache".snapshots = false;
    #kdn.disks.impermanence."usr/data".snapshots = true;
    #kdn.disks.impermanence."usr/state".snapshots = false;

    /* just a single impermanence example goes below */
    #kdn.disks.impermanence."usr/data" = {
    #  #zpool.name = cfg.zpool-main.name;
    #  #zfsPrefix = "${config.networking.hostName}/impermanence";
    #  #zfsPath = "${zfsPrefix}/${"usr/data"}";
    #  imp.directories = [
    #    "/var/lib/libvirt/images"
    #  ];
    #  imp.users.root.directories = [
    #    # ...
    #  ];
    #  imp.users.kdn.directories = [
    #    ".local/share/atuin"
    #    ".local/share/nix"
    #    ".local/share/containers"
    #    "dev"
    #  ];
    #};
    #environment.persistence."/nix/persist/usr/data" = {
    #  enable = true;
    #  hideMounts = true;
    #  users.root.home = "/root";
    #} // cfg.impermanence."usr/data".imp;
    #disko.devices.zpool."${cfg.zpool-main.name}".datasets."${cfg.impermanence."usr/data".zfsPath}" = { ... };

    #kdn.disks.tmpfs.size = "16M";
    #disko.devices.nodev."/" = { fsType = "tmpfs"; mountPoints = ["size=${cfg.tmpfs.size}" ... ]; };
    #disko.devices.nodev."/" = { fsType = "tmpfs"; mountPoints = ["size=${cfg.tmpfs.size}" ... ]; };
  }
)

see hosts/pwet for a different storage topology:
- boot disk also holds the LUKS header and volume,
- zvol is used as a LUKS header for the second zpool,

add all your required environment.persistence entries
run unified "deploy to install-iso" script utilizing nixos-anywhere and generating secrets automatically:
```
set HOST_NAME "pwet"
set HOST_CONNECTION "192.168.73.211"
nix run ".#kdnctl" -- host -n "$HOST_NAME" -a "$HOST_CONNECTION" bootstrap
```
- the script automates and therefore deprecates most of the follow-up steps
- TODO: try and document chroot using nixos-enter from the install-iso
- TODO: provide a "cleanup & reboot" script counterpart after bootstrap

former, deprecated steps (should be documented in kdnctl host bootstrap before removing):

set up keyfiles for each disk:

# this is fish shell
set HOST_NAME faro
set DISK_NAME virtual
dd if=/dev/random bs=1 count=2048 of=/dev/stdout | pass insert --force --multiline "luks/$DISK_NAME-$HOST_NAME/keyfile"

run nixos-anywhere to deploy

# this is fish shell
set HOST_NAME faro
set DISK_NAME virtual
set HOST_CONNECTION root@192.168.73.79
# build locally
nixos-anywhere --phases disko,install --disk-encryption-keys "/tmp/$DISK_NAME-$HOST_NAME.key" "$(pass show "luks/$DISK_NAME-$HOST_NAME/keyfile" | psub)" --flake ".#$HOST_NAME" "$HOST_CONNECTION"

# build on the target machine
nixos-anywhere --phases disko,install --build-on-remote --disk-encryption-keys "/tmp/$DISK_NAME-$HOST_NAME.key" "$(pass show "luks/$DISK_NAME-$HOST_NAME/keyfile" | psub)" --flake ".#$HOST_NAME" "$HOST_CONNECTION"

set up either of for each disk:

(unattended) TPM2 unlock:

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$HOST_CONNECTION" sudo systemd-cryptenroll --unlock-key-file="/tmp/$DISK_NAME-$HOST_NAME.key" --tpm2-device=auto "/dev/disk/by-partlabel/$DISK_NAME-$HOST_NAME-header"

(attended) YubiKey FIDO2 (touch required, without PIN) unlock:

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$HOST_CONNECTION" sudo systemd-cryptenroll --unlock-key-file="/tmp/$DISK_NAME-$HOST_NAME.key" --fido2-device=auto --fido2-with-client-pin=false --fido2-with-user-verification=false "/dev/disk/by-partlabel/$DISK_NAME-$HOST_NAME-header"

add SSH key to /.sops.yaml

run on over ssh: umount -R /mnt && zpool export -a && reboot now

partial recovery

decrypt volume using FIDO2:

set HOST_NAME faro
set DISK_NAME virtual
set HOST_CONNECTION root@192.168.73.79
set ENCRYPTED_DEVICE /dev/vdb
set ENCRYPTED_HEADER /dev/vda2
set ENCRYPTED_NEW_SLOT 3

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$HOST_CONNECTION" systemd-cryptsetup attach "$HOST_NAME-main-crypted" "$ENCRYPTED_DEVICE" - header="$ENCRYPTED_HEADER"

copying the keyfile to the machine:

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$(pass show "luks/$DISK_NAME-$HOST_NAME/keyfile" | psub)" "$HOST_CONNECTION:/tmp/$DISK_NAME-$HOST_NAME.key"

generate & register a passphrase:

pass generate "luks/$DISK_NAME-$HOST_NAME/passphrase" 32
# copy it to device
scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$(pass show "luks/$DISK_NAME-$HOST_NAME/passphrase" | tr -d '\n' | psub)" "$HOST_CONNECTION:/tmp/$DISK_NAME-$HOST_NAME.passphrase"
# add it to the header
ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$HOST_CONNECTION" cryptsetup luksAddKey --header="$ENCRYPTED_HEADER" --new-key-slot="$ENCRYPTED_NEW_SLOT" --key-file="/tmp/$DISK_NAME-$HOST_NAME.key" "$ENCRYPTED_DEVICE" "/tmp/$DISK_NAME-$HOST_NAME.passphrase"
# decrypt, paste the key into it
ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$HOST_CONNECTION" cryptsetup open --header="$ENCRYPTED_HEADER" "$ENCRYPTED_DEVICE" "$HOST_NAME-main-crypted"

Building on Hetzner Cloud from NixOS installer image

mount the NixOS installer image

run the build:

APPLY=1 HOST="<HOST>" bash <(curl -L 'https://raw.githubusercontent.com/nazarewk-iac/nix-configs/main/hetzner.sh')

Minimal NixOS bootstrap without custom installer

Used this snippet to gain initial access to nixos-avf (a NixOS VM on Android phone):

{ config, lib, pkgs, ... }:

{
  imports = [
    <nixos-avf/avf>
  ];

  avf.defaultUser = "kdn";

  system.stateVersion = "26.05";
  services.openssh.enable = true;
  services.openssh.ports = [2022 22];
  environment.systemPackages = with pkgs; [
    git
    vim
  ];

  services.netbird.clients.priv.port = 0;

  nix.settings = {
    show-trace = true;
    experimental-features = [
      "nix-command"
      "flakes"
    ];
  };
  nixpkgs.config = {
    allowAliases = true;
    allowUnfree = true;
  };

  users.users.kdn = {
    uid = 31893;
    linger = true;
    openssh.authorizedKeys.keys = [
      "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIFngB2F2qfcXVbXkssSWozufmyc0n6akKYA8zgjNFdZ ssh@kdn.im"
    ];
  };
}

Interaction between NixOS and Home Manager

https://jdisaacs.com/blog/nixos-config/

How to find out what uses the specific store path?

Find immediate parents/reverse dependencies: nix-store --query --referrers <paths...>.

Find the root using paths: nix-store --query --roots <paths...>.

Fix nix store errors

Fix errors like /nix/store/*-source not found: sudo nix-store --repair --verify --check-contents.

GitHub rate-limiting unauthenticated requests

see https://discourse.nixos.org/t/flakes-provide-github-api-token-for-rate-limiting/18609

add token to ~/.config/nix/nix.conf:

access-tokens = github.com=github_pat_XXXX

example of standalone Nix module

https://gist.github.com/nazarewk/8988facb6118f73d2db3d28b64463cba

systemd-cryptenroll --fido2-with-user-presence=false doesn't work with YubiKey

YubiKey's FIDO2 applet seems to ALWAYS require touch, so it cannot be used for unattended unlock of LUKS volume:

> systemd-cryptenroll --unlock-key-file=$(cat /tmp/emmc-etra.key | psub) --fido2-device=auto --fido2-with-client-pin=false --fido2-with-user-verification=false --fido2-with-user-presence=false /dev/disk/by-partlabel/disk-boot-emmc-etra-header
Initializing FIDO2 credential on security token.
👆 (Hint: This might require confirmation of user presence on security token.)
Generating secret key on FIDO2 security token.
👆 Locking without user presence test requested, but FIDO2 device /dev/hidraw1 requires it, enabling.
New FIDO2 token enrolled as key slot 1.

Keeping nixpkgs fork with arbitrary patches up to date

Using my own .flake.patches/update.py script:

Create required nixpkgs inputs:

nixpkgs-upstream - desired upstream nixpkgs branch
nixpkgs - the fork you have access to that will be managed by the updater
add entries to .flake.patches/config.toml

run nix run '.#nixpkgs-update' g:patches

Most of it is wrapped in /nixpkgs-update.sh and top entries of /flake.nix.

TODOs

TODO: evaluate https://github.com/Mic92/nix-fast-build
TODO: evaluate https://github.com/nix-community/nix-eval-jobs
TODO: try to integrate lanzaboote?

Name		Name	Last commit message	Last commit date
.agents		.agents
.claude		.claude
.flake.patches		.flake.patches
.idea/runConfigurations		.idea/runConfigurations
bpftrace		bpftrace
checks		checks
data		data
docs		docs
hack		hack
hosts		hosts
lib		lib
lsp/conf		lsp/conf
modules		modules
packages		packages
pic		pic
templates/terraform		templates/terraform
tools/kdnctl		tools/kdnctl
.gitignore		.gitignore
.sops.yaml		.sops.yaml
AGENTS.md		AGENTS.md
CLAUDE.md		CLAUDE.md
LICENSE		LICENSE
README.md		README.md
TASKS.md		TASKS.md
default.unattended.sops.yaml		default.unattended.sops.yaml
devenv.lock		devenv.lock
devenv.nix		devenv.nix
devenv.yaml		devenv.yaml
disko-build-scripts.sh		disko-build-scripts.sh
disks-format.sh		disks-format.sh
disks-mount.sh		disks-mount.sh
disks-recreate.sh		disks-recreate.sh
disks-umount.sh		disks-umount.sh
dns.sops.yaml		dns.sops.yaml
flake-update.sh		flake-update.sh
flake.lock		flake.lock
flake.nix		flake.nix
hetzner.sh		hetzner.sh
known-hosts.sh		known-hosts.sh
llms.nonsensitive.sops.yaml		llms.nonsensitive.sops.yaml
nix-build.sh		nix-build.sh
nix-compare-packages.sh		nix-compare-packages.sh
nixos-rebuild.sh		nixos-rebuild.sh
podman-nix-run.sh		podman-nix-run.sh
secrets-mount.sh		secrets-mount.sh

Navigation Menu

Uh oh!

Repository files navigation

nix-configs

Overview

Notes

Tasks

Custom ISO installer

Golden path for bootstrapping new physical machine

partial recovery

Building on Hetzner Cloud from NixOS installer image

Minimal NixOS bootstrap without custom installer

Interaction between NixOS and Home Manager

How to find out what uses the specific store path?

Fix nix store errors

GitHub rate-limiting unauthenticated requests

example of standalone Nix module

systemd-cryptenroll --fido2-with-user-presence=false doesn't work with YubiKey

Keeping nixpkgs fork with arbitrary patches up to date

TODOs

About

Used by

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

nix-configs

Overview

Notes

Tasks

Custom ISO installer

Golden path for bootstrapping new physical machine

partial recovery

Building on Hetzner Cloud from NixOS installer image

Minimal NixOS bootstrap without custom installer

Interaction between NixOS and Home Manager

How to find out what uses the specific store path?

Fix nix store errors

GitHub rate-limiting unauthenticated requests

example of standalone Nix module

systemd-cryptenroll --fido2-with-user-presence=false doesn't work with YubiKey

Keeping nixpkgs fork with arbitrary patches up to date

TODOs

About

Topics

Resources

Stars

Watchers

Forks

Used by

Contributors

Languages