NixOS Escape Hatch

March 20, 2024

I’ve used NixOS exclusively for several years now, and it’s occasionally useful to have an “escape hatch” into a different distro.

For certain tasks NixOS is more difficult than it needs to be, so being able to run a task in Ubuntu/Arch/etc has saved time by not having to deal with the Nix configs.

For example, I work with a lot of different software projects, and sometimes I want to use something that isn’t trivial to get setup in NixOS. Many projects will have build instructions for Ubuntu, and officially support building only on Ubuntu, and don’t yet have a nixpkgs package for NixOS.

In those cases figuring out the custom Nix shell environment can be a bit of a pain. And if I just want to jump in and fix a small bug it isn’t worth it to go through all that setup.

Enter Containers Link to heading

Instead of using docker, you can use the already builtin systemd-nspawn.

NixOS uses systemd as it’s init system, so it’s already installed and you just need to set it up. Systemd can manage a full container setup using systemd-nspawn and machinectl.

No installing docker if you don’t need it. No managing a docker-compose file. Just pull and install an image and you’re good.

Containers will be saved in /var/lib/machines

GUI Link to heading

It is possible to use GUI programs from inside the container. I don’t use them (yet), but it should be possible by setting the DISPLAY variable in the environment. See the docs on the Arch wiki for some pointers.

I will update this if I start using a GUI from the container.

Nix Config Link to heading

Here is the Nix code needed for an nspawn container named “ubuntu”. You should adapt this to any containers that you want to setup, feel free to make more than one. Before using this config you should setup the machine named ubuntu or you’ll run into some errors.

The code below bind mounts my code directory /home/patrick/code, since I use this environment for working with code projects that I don’t want to bother setting up in NixOS. You should adapt this to your system, and include whatever folders you want to share with the container.

If you want to see what any of the options do, look them up on NixOS Search

  systemd.targets.machines.enable = true;
  systemd.nspawn."ubuntu" = {
    enable = true;
    execConfig = {Boot = true;};

    filesConfig = {
      # Bind resolve.conf to get networking
      BindReadOnly = ["/etc/resolv.conf:/etc/resolv.conf"];
      # Bind any directories that you want to be shared
      Bind = ["/home/patrick/code"];
    };
    networkConfig = {Private = false;};
  };
  systemd.services."systemd-nspawn@ubuntu" = {
    enable = true;
    requiredBy = ["machines.target"];
    overrideStrategy = "asDropin";
  };

This sets up a systemd service that will manage the container and start at boot. Use the normal systemctl stop/start/restart commands to enable/disable the container service.

Installing Link to heading

To actually setup and use the machine, we’ll use nspawn.org. They host container images for several common distros which makes it very easy to get started. See their FAQ for their instructions, but the short version is that machinectl is what you should use for most everything.

NOTE: The command shown here pulls the image without verifying the signature. If you want to be extra cautious, follow the instructions on the nspawn FAQ to setup signature verification

> machinectl pull-tar https://hub.nspawn.org/storage/ubuntu/mantic/tar/image.tar.xz ubuntu --verify=no

Manage Link to heading

Using the machine is pretty easy. Start the container and then login with:

> machinectl start ubuntu
> machinectl login ubuntu

The default root login for the nspawn.org images is root/root.

machinectl has subcommands for stopping, deleting, or cloning the machine. I suggest you read through the options in machinectl --help.

One method you could use for it is to setup a base Ubuntu/Arch image and then clone it as needed for testing or specific projects. Or you could just have single long-running machines that you keep around.

With root access, I trust you can take it from here, so I won’t bore you with any more details.