Good Question; fair and concise… I’m Dan Dean.
+I’m working as a DevOps engineer, but I joke that a more accurate +title would be DevSecRelFinOps Engineer & SRE (I bill by the syllable). +My team and I solve all of the behind-the-scenes, cloud-native issues that we +face at our company to keep the applications running securly and reliably in the cloud. +In my free time I enjoy learning new technologies. +Sometimes I write about my findings in my blog.
+ +I set my home lab is set up with K3s. This is great, but I’d like to access it from not just inside my dungeon. +To make this happen, I set up the network with Tailscale. With a NixOS, this is dirt-simple: +{ lib, config, ... }: { options = { tailscale.enable = lib.mkEnableOption "Enable Tailscale"; }; config = { services.tailscale.enable = config.tailscale.enable; }; } I added the Tailnet IP to the ~/.kube/config. +apiVersion: v1 clusters: - cluster: server: https://<Tailnet IP>:6443 name: default kubectl get nodes… +...
+I ran out of space on my 2020 Macbook Pro M1. I’m not sure what it says about what kind of person I am, but to me that just meant I had to factory reset it. And if I was factory resetting it, I might as well boot it with Linux. +I spent some time (less than a week) on the Asahi Fedora Remix. Before this, I was using Nix Darwin, but Nix isn’t available on Fedora. +...
+Good question.
+I’m hoping to write down some of my thoughts on computers as I white-knuckle it into the future.
+Recently I’ve been interested in the following:
+Shout out to Eyal Kalderon for the Zola website theme.
+ + +I set my home lab is set up with K3s. This is great, but I’d like to access it from not just inside my dungeon. +To make this happen, I set up the network with Tailscale. With a NixOS, this is dirt-simple: +{ lib, config, ... }: { options = { tailscale.enable = lib.mkEnableOption "Enable Tailscale"; }; config = { services.tailscale.enable = config.tailscale.enable; }; } I added the Tailnet IP to the ~/.kube/config. +apiVersion: v1 clusters: - cluster: server: https://<Tailnet IP>:6443 name: default kubectl get nodes… +...
+I ran out of space on my 2020 Macbook Pro M1. I’m not sure what it says about what kind of person I am, but to me that just meant I had to factory reset it. And if I was factory resetting it, I might as well boot it with Linux. +I spent some time (less than a week) on the Asahi Fedora Remix. Before this, I was using Nix Darwin, but Nix isn’t available on Fedora. +...
+I set my home lab is set up with K3s. +This is great, but I’d like to access it from not just inside my dungeon.
+To make this happen, I set up the network with Tailscale. +With a NixOS, this is dirt-simple:
+{ lib, config, ... }: {
+ options = { tailscale.enable = lib.mkEnableOption "Enable Tailscale"; };
+ config = { services.tailscale.enable = config.tailscale.enable; };
+}
+I added the Tailnet IP to the ~/.kube/config.
apiVersion: v1
+clusters:
+- cluster:
+ server: https://<Tailnet IP>:6443
+ name: default
+kubectl get nodes…
Unable to connect to the server: tls: failed to verify certificate: x509: certificate is valid for <Internal IP>
+not <Tailnet IP>
+D: +Lets make sure we can actually reach the nodes over the Tailnet…
+
+$ k get nodes --insecure-skip-tls-verify
+NAME STATUS ROLES AGE VERSION
+leonardo Ready <none> 9d v1.34.1+k3s1
+raphael Ready <none> 20d v1.32.4+k3s1
+splinter Ready control-plane,etcd,master 95d v1.32.7+k3s1
+Yeah seems like we can.
+After a bit more reading, it looks like we need to rotate the cert for the new IP +Luckily there’s nothing important running on this cluster!
+Let’s just restart the master node… And nothing.
+After a bit more digging, this issue looks promising Remote Kubectl
+We need to edit the k3s-serving secret in the kube-system to include the new Tailnet IP of the master node.
kind: Secret
+metadata:
+ annotations:
+ listener.cattle.io/cn-<Internal IP>: 10.0.0.10
+ listener.cattle.io/cn-<Tailnet IP>: 10.43.0.1
+After modifying the cluster secret and k3s regenerate certificates, we are back in business.
$ k get nodes
+NAME STATUS ROLES AGE VERSION
+donatello Ready <none> 8d v1.32.4+k3s1
+leonardo Ready <none> 30d v1.34.1+k3s1
+raphael Ready <none> 41d v1.32.4+k3s1
+splinter Ready control-plane,etcd,master 116d v1.32.7+k3s1
+Is this reproducible? Probably not. +Since the certs are already generated for the master node, I didn’t feel like going through the exercise of completely bootstrapping the cluster again to test a reproducible setup with the new IP. +To test this, we could add the following to the cluster settings.
+if config.isK3sNode.isServer then [ "-tls-san=${hostTailIP}" ] else [ ];
+I ran out of space on my 2020 Macbook Pro M1. +I’m not sure what it says about what kind of person I am, but to me that just meant I had to factory reset it. +And if I was factory resetting it, I might as well boot it with Linux.
+I spent some time (less than a week) on the Asahi Fedora Remix. +Before this, I was using Nix Darwin, but Nix isn’t available on Fedora.
+The folks in the nix community put together some solid documention about booting NixOS on Apple Silicon. +I figured it was worth a shot.
+The only significant change that I made was the initial install. +Since I had already installed Fedora, I figured I could just install NixOS right onto that disk partition.
+I built the ISO from scratch, made some configuration changes as per the nixos-apple-silicon docs, & installing the OS went sideways…
During the initial install, my machine died.
+Forgot to plug it in.
+Luckily, all of the changes were already written to disk, so I could just install from the /etc/nixos/configuration.nix file after mounting the disk again.
+After some time, the install went through fine.
Restart… And nothing :(
+More specifically, I was stuck in the boot menu. +Naturally, this was caused by me going off-script, installing NixOS over the Fedora install.
+Now I’ve been a Linux enjoyer for while, but I haven’t played much with any boot menus. +From what I can tell, however, I needed to point to the new /boot partition I made for NixOS. +Simple enough.
+In U boot, we can printenv, which tells us everything we need to know.
The docs for the bootflow command can be found here
The bootcmd was bootflow scan -b.
+Hmmm…
With a bootflow list, we can see what we’re working with:
Seq Method State Uclass Part Name FileName
+-------------------------------------------------------------------------------------------
+0 efi_mgr ready (none) 0 <NULL>
+1 efi ready nvme 4 nvme@27bccc000.blk#1.boot /EFI/BOOT/BOOTARG64.EFI
+-------------------------------------------------------------------------------------------
+This looks promising. This issue is, it doesn’t mean anything to me.
+It seemed clear that the <Null> and missing FileName were issues.
+So I tried booting off of the first bootflow and that did the trick.
bootflow select 1
+bootflow boot
+So it seems like we needed to boot the 1st bootflow instead of the 0th.
+This aligns with the docs for the -b option:
-b
+
+ Boot each valid bootflow as it is scanned. Typically only the first bootflow matters, since by then the system boots in the OS and U-Boot is no-longer running. bootflow scan -b is a quick way to boot the first available OS. A valid bootflow is one that made it all the way to the loaded state. Note that if -m is provided as well, booting is delayed until the user selects a bootflow.
+That means we needed to change the bootcmd to the following:
+setenv bootcmd "bootflow scan ; bootflow select 1 ; bootflow boot"
+Then saveenv, naturally.
I also have a 2020 Mac Mini with the M1 chip. +I figured I can’t have just one of them on Asahi & the other on MacOS, that’d be cruel. +Also, I already had the bootable USB for NixOS on Asahi.
+This time around it was much smoother because there wasn’t an existing Fedora partition.
+Thanks to the reproducibility of Nix, I was able to reuse the majority of my configuration from the previous host. +The code for this can be found in my nixos-config repo.
+ + +