Skip to content

Your first image flake

This page is for an image author who builds a cluster's images with Nix. At the end you have a flake that builds unsigned images into the directories the cluster's inventory names, such as images/node-metal, and you know what the build writes there and what else mkImages returns. The page follows the homelab example: images for bare metal, one for its router and one that its three Kubernetes nodes run.

Before you begin

You need Nix with flakes enabled on an x86-64 Linux machine, and a git repository for the flake. A flake sees only the files git tracks, so git add a new module before you build.

Nix is needed only on the machine that builds images. chalkctl and chalklab never evaluate the flake: the inventory names the directory a build was linked to, so the machine that runs chalkctl needs that directory and no Nix.

The flake calls mkImages

The homelab's flake builds a router image, a Kubernetes node image and the installer.

flake.nix
{
  description = "A chalkos home lab: a router and three Kubernetes nodes on an IPv6 fabric";

  inputs.chalkos.url = "github:trevex/chalkos/v0.1.0"; # x-release-please-version

  outputs =
    { self, chalkos, ... }:
    let
      system = "x86_64-linux";
      pkgs = chalkos.inputs.nixpkgs.legacyPackages.${system};
      # The router and Kubernetes node images use their layers' built-in BGP.
      images = chalkos.lib.mkImages {
        # The commit the images are built from, which chalkctl shows beside an image's root hash.
        revision = self.rev or self.dirtyRev or null;
        images = [
          {
            name = "router-metal";
            platform = "metal";
            modules = [ chalkos.nixosModules.router ];
          }
          {
            name = "node-metal";
            platform = "metal";
            modules = [ chalkos.nixosModules.kubernetes ];
          }
        ];
      };
    in
    {
      # nix build .#<image> -o images/<image>: the images where the inventory, chalkos.yaml,
      # points. nix build .#installer builds the raw image and ISO that install them.
      packages.${system} = {
        inherit (images)
          router-metal
          node-metal
          installer
          ;
      };

      # The images' evaluated NixOS systems, whose build outputs a guide builds on their own, such
      # as an image's schemas:
      # nix build .#nixosConfigurations.node-metal.config.system.build.chalkosSchemas
      nixosConfigurations = chalkos.lib.nixosOf images;

      # nix flake check: the images' tests, chalkos.tests.
      checks.${system} = chalkos.lib.testsOf images;

      # nix develop: chalkctl and chalkbuild of the chalkos this flake pins, and the other tools
      # the guide runs.
      devShells.${system}.default = pkgs.mkShellNoCC {
        packages = [
          chalkos.packages.${system}.chalkctl
          chalkos.packages.${system}.chalkbuild
          pkgs.kubectl
          pkgs.jq
          pkgs.openssl
          pkgs.wireguard-tools
          pkgs.arp-scan
          pkgs.netcat
          # The air-gapped lab copies the cluster's container images with these.
          pkgs.crane
          pkgs.yq-go
        ];
      };
    };
}

The flake takes chalkos as an input and calls chalkos.lib.mkImages. flake.lock pins the chalkos version, so the images change only when the lock file or the flake's own modules change. The call sets two arguments:

  • revision is the commit the images are built from. chalkctl shows it beside an image's root hash.
  • images lists the images to build, each with its name, its platform and its NixOS modules. All are for bare metal. router-metal lists the router layer, chalkos.nixosModules.router. node-metal lists the Kubernetes layer, chalkos.nixosModules.kubernetes. Both layers provide their own BGP speaker.

A layer is a predefined set of NixOS modules that chalkos ships, listed like any other module. An image lists one layer at most, and an image without one runs chalkos's base alone.

mkImages returns each image by its name, such as images.node-metal, and the installer as images.installer. The flake exposes them as packages of the same names. chalkos.lib.testsOf collects each image's tests, which the flake exports as its checks, so nix flake check runs them, as Test an extension describes. The flake's shell, which nix develop enters, holds chalkctl and chalkbuild of the chalkos that flake.lock pins. One image serves every node the inventory gives it. What differs between nodes, from their addresses to the cluster's endpoint, is their config: YAML documents that chalkctl delivers, which Config and the inventory describes.

mkImages checks its arguments as module options

mkImages takes these arguments:

Argument Default What it sets
images [ ] The images to build, each with its name, platform and NixOS modules
modules [ ] NixOS modules every image carries, before its own
osCA null The public part of the cluster's secrets, whose OS CA the images and the installer trust in maintenance mode
installer.modules [ ] The installer's own NixOS modules
nixpkgs the nixpkgs chalkos is tested against The nixpkgs the images are evaluated and built from
system x86_64-linux The platform the images are built for, as Nix names it
specialArgs { } Extra arguments for every image's modules, such as a flake's inputs
revision null The git revision the images are built from
tag null The git tag the images are built from, such as a release's

The arguments are module options, so a misspelled argument fails the evaluation and names the argument. The mkImages reference documents each one. Images and platforms covers images, modules and osCA. The homelab's flake sets no osCA, which suits only a lab on a private network. The home lab guide adds it in the step that generates the secrets.

revision and tag say where an image came from. Nix cannot read git tags, so whatever builds a release passes the tag:

images = chalkos.lib.mkImages {
  revision = self.rev or self.dirtyRev or null;
  tag = "v1.5.0";
  images = [
    {
      name = "node-metal";
      platform = "metal";
      modules = [ chalkos.nixosModules.kubernetes ];
    }
  ];
};

self must be among the arguments of the flake's outputs, as in the homelab's flake. Each image directory's metadata.json records both values, and chalkctl shows them beside the root hash's first 12 hex characters, as in 3fa9c1e2b7d0 (v1.5.0, a1b2c3d4e5f6). They never reach the image's store, so they change no root hash, and nothing orders or refuses an image by them.

An image has no version to raise. Its identity is the root hash of its store: a changed module changes the store, so the build is a new image. The build refuses system.image.version, because systemd-boot would order boot entries by it.

Build the image

Build each package and link the result where the inventory looks for it:

$ nix build .#router-metal -o images/router-metal
$ nix build .#node-metal -o images/node-metal
$ ls images/node-metal
chalkos.raw
facts
metadata.json
repart.d
repart-output.json
schemas

images/node-metal is a link into /nix/store. The homelab's inventory names it as the image node-metal and gives it to the nodes of the group nodes, and gives router-metal to the router:

chalkos.yaml
apiVersion: chalkctl.chalkos.dev/v1alpha1
kind: Inventory
metadata: {name: homelab}
spec:
  images:
    router-metal: {source: {directory: images/router-metal}}
    node-metal: {source: {directory: images/node-metal}}
  groups:
    router: {imageName: router-metal, files: [router/*.yaml]}
    nodes: {imageName: node-metal, files: [cluster/*.yaml, bgp/fabric.yaml, bgp/secrets.yaml]}
  nodes:
    router: {address: 192.168.1.2, groups: [router]}
    node1: {address: "fdc8:a1b2:c3d4:ff::11", groups: [nodes], files: [nodes/node1.yaml]}
    node2: {address: "fdc8:a1b2:c3d4:ff::12", groups: [nodes], files: [nodes/node2.yaml]}
    node3: {address: "fdc8:a1b2:c3d4:ff::13", groups: [nodes], files: [nodes/node3.yaml]}

A relative directory is relative to the inventory's directory. The next build replaces the link, so the inventory names the newest image without an edit.

The image directory holds the disk image and its contract

The build writes six entries:

  • chalkos.raw is the raw disk image. Its slot A carries labels named after its root hash, which the build sets once systemd-repart has computed the hash.
  • repart-output.json is systemd-repart's report of the partitions, with their offsets, sizes and the root hash. chalkctl reads the store, the hash tree and the UKI at those offsets.
  • repart.d holds the partition definitions of the system region, which the installer lays out on a target disk.
  • metadata.json names the image's root hash, name and platform, the revision and tag it was built from, and the kinds it accepts.
  • schemas holds the JSON Schema of each kind the image accepts.
  • facts holds a <layer>.json for each of the image's layers that has facts. On an image with Kubernetes, facts/kubernetes.json records what the image fixes about Kubernetes: the release of its kubelet and its pod network. On an image without such layers, facts is empty.

chalkctl reads metadata.json, schemas and facts when it validates, installs or upgrades a node whose image in the inventory is this directory. Image and partition layout lists the files in detail.

The image is unsigned. A node that does not enforce Secure Boot installs it as it is. Nodes that enforce Secure Boot need a signed copy, which Sign images for Secure Boot makes on a machine that holds the key, so the key never enters /nix/store.

chalkos does not wrap image conversion. qemu-img convert -f raw -O qcow2 makes a qcow2 disk for KVM, zstd compresses the raw image for transport, and dd or a USB writer puts it on a disk.

An image carries its OCI layout, NixOS system and tests

The homelab exposes three images and the installer by name. Each image mkImages returns is its image directory, and carries the other forms of the image as attributes. Each is built only when something asks for it:

flowchart LR
    flake["flake.nix and flake.lock"] -->|calls| mk["mkImages"]
    mods["Your NixOS modules"] -->|arguments| mk
    mk --> image["images.node-metal"]
    mk --> installer["images.installer"]
    image -->|nix build| dir["Image directory"]
    image -->|.oci| oci["OCI layout"]
    oci -->|chalkbuild push| registry["Registry"]

The diagram shows the flake and its modules passing through mkImages into each image and the installer. The image directory is what the inventory names; the OCI layout goes to a registry.

  • images.node-metal is the image directory. Its name and platform are the entry's, and its layer is "kubernetes", "router" or null.
  • images.node-metal.oci is the same image as an OCI image layout, as chalkbuild oci writes it. Publish images to a registry pushes it.
  • images.node-metal.nixos is the evaluated NixOS system. It reads the image's options, such as images.node-metal.nixos.config.chalkos.kubernetes.package.version, and its config.system.build holds parts of the image that build on their own, such as its schemas, chalkosSchemas.
  • images.node-metal.tests builds the image's tests, its chalkos.tests.
  • images.installer is one directory with a raw image and a hybrid ISO, which installs images of every platform. Customise the installer adds to it.

chalkos.lib has helpers that map over the images and leave the installer out. ociOf, nixosOf and testsOf return each image's oci, nixos and tests by its name. selectImages picks the images whose platform or layer match, so a flake can expose every Kubernetes image as a package:

packages.x86_64-linux = chalkos.lib.selectImages { layer = "kubernetes"; } images;

chalkos.lib is the library a flake calls, with mkImages and these helpers. Image modules get another one as a module argument, chalkosLib, with helpers such as types, mkOption, render, validate and reloadOn.

The homelab's flake exports the NixOS systems as nixosConfigurations and the tests as its checks:

nixosConfigurations = chalkos.lib.nixosOf images;
checks.${system} = chalkos.lib.testsOf images;

The guides use the NixOS systems to build one part of an image without the rest, such as its schemas with nix build .#nixosConfigurations.node-metal.config.system.build.chalkosSchemas.

Check that it worked

chalkctl validate checks every node's config files against the schemas of the image the inventory names for it. It reaches no node and needs no secrets:

chalkctl validate

For the homelab it prints a line per node, with the image's root hash and the hash of the node's config:

node1: valid for the image node-metal (node-metal on metal, root hash 3b80d0bf1243), config 451b547e8373fe70683c9dbf95f7cc08916ae0e920702da6c1a2c3e85c6f4080
node2: valid for the image node-metal (node-metal on metal, root hash 3b80d0bf1243), config 10c35f7fe988679c3a54691ddf95e78267269d22297fb360ceae5527f5965d50
node3: valid for the image node-metal (node-metal on metal, root hash 3b80d0bf1243), config 55ed0b193224383f68d0bb4014e45885697279cd67dfa6fc9232627757cfbd44
router: valid for the image router-metal (router-metal on metal, root hash cd7ac91cb655), config dc6178bbae55b1fe0e6af7b10502a1099f708c995b7be242ee95bb92050f87e6

A document of a kind the image does not declare, or a misspelled field, fails here. The image is then ready to install on a machine with Install a node, or to roll out to running nodes with Upgrade nodes.

What next