Posts

I found it a bit fiddly to get a Xbox controller I had used for years on Fedora working on NixOS. I recall all sorts of incantations to get it working on Fedora and thankfully it was a bit easier on NixOS, but I had to piece together some reddit posts and other content to get it working. TL;DR Add this to your configuration: hardware.steam-hardware.enable = true; hardware.xone.enable = true; environment.systemPackages = with pkgs; [ linuxKernel.packages.linux_zen.xone ]; Reboot Plug in adapter Hold sync on controller Hold sync on adapter That seemed to do the trick to get it working on Steam. ...

I started this post in April 2024 but never committed it to the repository. I’ve recently dug into K8s and am strongly considering migrating my lab over- so I figured I should finish this up as gitops upgrades for virtual machines has been reducing time I spend on updates for about 1 year. Reliability A core principle in my lab has always been reliability. Many design choices may appear outdated, inefficient, expensive, or bespoke, but are intentional in my desire to maintain reliability and simplicity in updates, backup restoration, availability, and ongoing maintenance. There are a single points of failure (many, actually) but fixing them is trivial save for a full non-disk hardware failure, which I hope to better solve with K8s and ceph (or OpenEBS- TBD). ...

Landing Headfirst The foray into NixOS was painful, and while there wasn’t much I found for newcomer guidance here’s what would have helped me: Start small, use the installer on a real system or VM to build on Said differently, don’t start with a beefy configuration that will inevitably require troubleshooting Yes, this is obvious advice, but I could not resist the allure of creating a “perfect” configuration As of 2023-11, the installer did not play well with LUKS It is worth learning how to manually partition disks properly Make /boot large (5GB+), or don’t put it on a dedicated partition I ended up needing to wipe and re-partition due to a full boot partition; a well documented issue that shockingly isn’t resolved in the installer When using -p to name generations, don’t use spaces nixos-rebuild will happily give you the rope, invalid generations won’t be cleaned out by garbage collection and you’ll find yourself unable to upgrade with a full /boot The following are a bit tongue-in-cheek that I have yet to do them as of this rambling: ...

I recently had a painful migration from Fedora -> Debian (temporary) -> NixOS on my primary workstation. My git/Ansible workflow is currently executed locally; it turns out Ansible/Python/NixOS can be a painful combination. There are a few others with similar issues. PEBCAK is not entirely ruled out in this scenario. From what I could tell, the core issue was getting Ansible to recognize the Python binary who could import the correct packages (installed via Nix config). ...

Background I run 2 Proxmox hosts, one being a primary and the secondary being a Proxmox Backup Server target. As is too common the primary is far different from the secondary including the boot configuration, which was the affected component in the latest Proxmox major upgrade (v7 to v8). The situation was further exacerbated by a “temporary” implementation of virtualized OPNsense after a failure of pfsense hardware in 2021. This makes the primary Proxmox host a rather juicy single point of failure. ...

16 data disks and counting I have a lot of data- the 12 3.5" slots on the storage primary host are all filled with the exception of a cold spare and a 20TB disk I connect quarterly to update the offline backup. Most of the data is fairly unique- personal pictures/videos, backups of old websites, images of prior computers (one day I’ll virtualize them for fun), tons of (actual) Linux ISOs (I keep a copy of almost all I use), and more. I’m a frequent visitor to the Internet Archive and like to keep personal copies of things. ...

I occasionally get a bit of ridicule when I describe my Homelab architecture. It’s not Kubernetes, built on a cloud provider, or all that exciting. I have convictions for my self-hosting which greatly influences some of my choices. In time I plan to modernize and shore up gaps with better management/automation enabled by Ansible. For now- it has worked so well for many years it’s been a tough pill to swallow on adding the complexity for the ~15 services I run (most are “production” for myself or my family). This may be fun to look back at 5 years from now assuming I’ve delivered on my intent to modernize. ...

In an effort to modernize my workflows I’ve finally started to add roles to my growing collection of Ansible playbooks. The first iteration was to remove some simple copy past I’ve done to make rolling back upgrades simple and nearly perfectly reliable. The previous workflow would be identifying risky updates, shutting down the VM, run a ZFS snapshot, start it up, then run the upgrade. Rolling back was simply reverting the snapshot and starting the VM- easy and almost impossible to screw up. The core issue was that my collection of services has grown to 20+ which means manual processes just didn’t scale. ...

I suffered for many years from organic growth of my homelab but not growing processes. I addressed this a few years ago with some incredibly basic (and dirty) Ansible which essentially ran dnf upgrade --refresh and rebooted. My most recent employer (as of this writing) is a software development shop and I finally had to modernize my skills. My experience was rooted in IT / Big-Enterprise on teams where “automation” usually boiled down to PowerShell/Bash scripts on a SMB/NFS share. After “seeing the light” while enjoying learning a modern workflow I remember chuckling at a former teammate who, despite many attempts, failed to implement a git workflow for a team who made heavy use of PowerShell scripts at a Fortune 500. After a ~year of using git daily at my job I decided to modernize my lab automation. ...

Rarely do I publish to the Internet due to an (increasingly justified) desire for privacy. Maybe these words will end up training a language model somewhere. They will almost certainly be mirrored on archives. I dislike that, but I suppose it’s a tradeoff. However, in an attempt to share information I’ve started with hugo. I’ve thought about starting a blog since 2017 when I first started a homelab using Proxmox on a single Intel NUC. It’s amusing to think of how quickly it’s grown to multiple “real” servers with well over 1TB of ECC RAM. ...