Skip to content

Glossary

Each entry defines a term the documentation uses repeatedly and links to the page that explains it. The first sentence of an entry is a short definition, which every page shows as a tooltip on the term. Option, command and API names link to the generated reference.

A/B

The update scheme of a node's disk, with two slots, A and B, that each hold an image. The node boots one slot while an upgrade writes the other, so the image it ran before stays on disk for a rollback. See The image.

Admin

The client role that may call almost every method of the node API. It may call every method but the one nodes renew their certificates with, which needs the role node: install, config changes, volume resets, bootstrap, etcd membership changes and rotations among them. See Security model.

Blessed boot

A boot that the health check found healthy. After it, systemd-bless-boot removes the boot counter from the UKI's file name. Nothing rolls a blessed image back on its own. See Boot, health and rollback.

Boot counting

systemd-boot's count of the tries a newly installed image has left. The count is kept in its UKI's file name (chalkos_<generation>+3.efi). Each boot that is not blessed uses a try; with none left, systemd-boot boots the previous image. The number of tries is chalkos.upgrade.bootTries. See Boot, health and rollback.

Bootstrap

The one-time start of a cluster's etcd on its first control plane. It runs with chalkctl bootstrap and is refused on a node that is bootstrapped or holds etcd data, so a cluster is never initialised twice; further control planes join on their own. See The control plane.

chalkbuild

The command that prepares the images Nix builds for a cluster's nodes. chalkbuild sign signs a copy of an image directory with the Secure Boot db key, on the machine that holds the key, and checks the copy against the db certificate. It never talks to nodes. See its commands and Sign images for Secure Boot.

chalkcfg

The command on every node that prints the node's config documents. It reads them from /run/chalkos/config and prints them by kind and name, as names, JSON, YAML, a jsonpath expression or a Go template. It has no network access and never prints Secrets or the documents of withheld kinds, such as Addons. An image's NixOS modules build its commands with chalkos.kinds.<Kind>.get, which checks a path against the kind's options when the image is evaluated. See its commands.

chalkctl

The command-line client of chalkos. It reads a cluster from an inventory and the image directories it names for the nodes, installs, upgrades and operates nodes through chalkd, and issues certificates from the secrets file. It never builds or signs images or runs Nix. See its commands and Architecture.

chalkd

The agent on every node. It serves the node API on TCP port 50000 over mutual TLS, installs and upgrades the node, applies its config, runs the Kubernetes control plane and decides whether a boot is healthy. See Architecture.

chalklab

The command that runs a cluster's kvm nodes as QEMU virtual machines on one machine. Each virtual machine has Secure Boot and a TPM, and chalklab installs and bootstraps them. See its commands and the quick start.

Change rule

A CEL expression that a config must make true for the image to accept it. An image's schemas declare it on a kind or on one of its fields. A rule that is false refuses the config, or says it takes chalkctl storage reset or a reinstall. See Guard changes with rules and facts.

Client file

A JSON file that lets someone operate nodes without the secrets file. It holds a client certificate of one client role, its key and the OS CA certificate, and chalkctl client new issues it. It names no cluster: its OS CA ties it to one, and a node of another cluster fails the TLS handshake with it. See Set up your workstation and Security model.

Client role

The permission a client certificate grants on the node API: reader, operator or admin. The role is taken from the certificate's Organization, and each role is allowed what the roles before it are. A certificate issued through the node CA has the role node, which may only renew its node certificate. See Security model.

Cluster endpoint

The URL of the cluster's API server, which kubelets, kubeconfigs and joining control planes use. It is spec.endpoint of the Cluster document. Nodes also renew their node certificates at port 50000 of its host, so it points at a control plane, a VIP or a load balancer that forwards both ports. See Kubernetes networking.

Cluster registry

A container registry on a Kubernetes cluster's control planes, which containerd on every node asks first for the images of the hosts it mirrors. The ClusterRegistry document lists its endpoints, and chalkctl registry push fills each of them, since they do not copy images to each other. See Run an air-gapped cluster with the cluster registry.

Config

A node's configuration at runtime: a set of config documents. Each document is of a kind its image accepts, such as Node, Network, SystemDisk, Cluster or an extension's own. The inventory names the YAML files whose documents form a node's set. chalkctl checks the set against the schemas of the node's image and delivers it at install and with chalkctl config apply; chalkd checks it again, keeps it on STATE and applies it at every boot. A new set replaces the node's previous one as a whole. Secrets travel as Secret documents and reach only the services that read the kinds referencing them. See Config and the inventory.

Config document

One object of a node's config, written in YAML like a Kubernetes object. It has an apiVersion of a group and a version, a kind, a metadata.name and a spec. The schema of its kind defines the spec; Config kinds lists the fields of the built-in kinds. On the node, chalkd writes each document except Secrets and those of withheld kinds, such as Addons, to /run/chalkos/config/<group>/<Kind>/<name>.json. See Config and the inventory.

Config slot

The directory on STATE that holds a node's config for one image, named by the image's root hash. It is /state/config/<root-hash> on STATE, and holds the documents as delivered, the same documents with the schemas' defaults filled in, the config's version and the node's identity. Each image's boot loads its own slot, so a rollback boots the previous image with the config it ran. An install or an upgrade writes the slot of the image it writes, and chalkctl config apply that of the running image. An upgrade removes the slots of images no UKI on the ESP boots any more, so a node holds at most two. See Config and the inventory.

Contract

What an image accepts as config: its name, platform, kind schemas and what it fixes about Kubernetes. The name and platform are the ones the image was built with, and the schemas are the JSON Schemas of its kinds. The image carries the schemas in /etc/chalkos/schemas and the Kubernetes facts in /etc/chalkos/kubernetes/image.json, and its image directory has them as schemas/ and facts/kubernetes.json. chalkctl reads the contract from the image directory when it validates config, installs from the installer or upgrades a node, and from the running node with the Contract method when it applies config. On an upgrade, the node reads it from the image it wrote. See Image and partition layout.

Control plane

A node that runs etcd and the Kubernetes API server, controller-manager and scheduler. Its Node document sets spec.type: controlplane. It runs those components as static pods that chalkd renders, besides containerd and the kubelet, and it is the only kind of node chalkctl delivers Addon documents to. See The control plane.

db and dbx

The UEFI firmware's Secure Boot databases: db lists what the firmware boots, dbx what it refuses. Both hold certificates and hashes of binaries. A chalkos image boots under Secure Boot when its signer's certificate is in db and nothing of it is in dbx. See The image.

Diagnostic

A read-only command of an image that chalkd runs on a node for a client, such as ip route get. Its image declares it in chalkos.diagnostics, with typed parameters and the least client role that runs it, and chalkd runs it in a transient systemd unit with a sandbox of its own. See chalkctl node diag.

dm-verity

The kernel's block-level integrity check, which makes a modified block of the store fail to read. Each block of the store is hashed into a hash tree whose top is the root hash, and a block that does not match its hash fails to read. See The image.

Effect

An action on the node's services after a delivery is applied, such as a unit restart. An effect restarts or reloads units, runs a command or runs a layer's action in chalkd, as chalkos.effects declares it. It fires when one of its outputs or the documents of one of its kinds changed, or when its CEL rule, like a change rule, holds. See React to changes.

ESP

The EFI system partition, a FAT file system at the start of the system disk. It holds systemd-boot and the UKIs of both slots, and is 1 GiB by default. See Image and partition layout.

etcd member

A control plane that belongs to the cluster's etcd. It is a learner while it catches up and a voter after. A control plane pins its addresses on STATE when it bootstraps, or before it adds itself as a learner, and every later boot waits for exactly them; chalkctl etcd leave undoes both. See The control plane.

Extension

NixOS modules outside chalkos that add services to an image and may declare config kinds. An extension sits in an image's modules and declares its kinds under chalkos.kinds, whose documents a node's YAML files then carry. The services read those documents through a grant or as outputs rendered from them, and effects restart them when they change. chalkos's own Kubernetes support declares its kinds and effects the same way. See Declare a config kind.

Fact

Node state that a change rule reads. Facts are declared in chalkos.facts: the JSON of a file, such as the Kubernetes share, or of a program of the image's store, which chalkd runs in a sandbox. See Guard changes with rules and facts.

Fallback keyslot

The second keyslot of every encrypted partition, which unlocks it when the TPM does not. It holds the node's recovery key by default, or a password, or the partition has none, as spec.encryption.fallback of the node's SystemDisk chooses. See Storage and encryption.

Generation

The number in a UKI's file name that orders a node's boot entries, the highest first. A node counts it itself: the file name of a UKI on the ESP is chalkos_<generation>.efi. The image and an install write generation 1; each upgrade writes the one above the highest on the ESP, so systemd-boot, which orders a node's UKIs by file name, boots the image installed last first, older or newer. See Boot and the image.

Grant

The config kinds a unit may read at runtime. A unit's grant is declared in systemd.services.<unit>.chalkos.kinds: chalkd writes the documents of those kinds below /run/chalkos/config and the values their secretRef fields name into the unit's credentials. A grant restarts nothing; an effect does. See Read config in a service.

Health check

The check that decides whether a counted boot is healthy. chalkos-health.service runs it. chalkd must serve on every node, and on a node with Kubernetes the preparation must have succeeded. Then a control plane that is an etcd member needs its local etcd member, its API server and its kubelet healthy, and a worker its kubelet healthy and its Node registered. A node without Kubernetes, or a control plane not bootstrapped yet, needs its config applied and no unit of the boot failed. A boot that passes within chalkos.upgrade.healthTimeout is blessed. See Boot, health and rollback.

Identity

The secret-free form that chalkd translates a node's config into before applying it. It holds the node's hostname, static addresses, labels, taints, storage and Kubernetes addresses. chalkd keeps it on STATE in each image's config slot, translated with that image's schemas, and the image's boot applies it. chalkctl validate translates a set the same way to check it. See Image and partition layout.

Image

The whole operating system of a node as one unsigned raw disk image. It has the ESP and slot A filled and is built by chalkos.lib.mkImage, or chalkos.lib.mkImages for a list of images, from a name, a platform and NixOS modules, a layer among them. Its name is CHALKOS_IMAGE in its os-release and name in its metadata.json, and chalkctl upgrade --image gives it to the nodes that run an image of that name. Every node that runs it boots the same bytes; what differs between nodes travels in their config. Its build output is an image directory. See The image.

Image directory

The build output of an image: the raw disk image with its metadata and config schemas. It is what chalkos.lib.mkImage returns, such as images.node-metal of chalkos.lib.mkImages, and holds the raw image, systemd-repart's report of its partitions, the definitions of the system region, metadata.json, which names the image's name, platform, root hash, provenance and kinds, the kinds' JSON Schemas in schemas/ and the facts of its layers in facts/. An image with Kubernetes has facts/kubernetes.json, which names the release of its kubelet and its pod network. The inventory names the image directory in an image's source.directory, chalkbuild signs a copy of it, and chalkctl reads it to validate config, install and upgrade. See Image and partition layout.

Installer

An image that always boots into maintenance mode and writes a node's image to a disk of the machine. chalkos.lib.mkImages builds one, as a raw image and a hybrid ISO, with installer.modules; it installs images of every platform. See Architecture.

Inventory

The file that tells chalkctl about a cluster: its nodes, their addresses, images and config files. It lists each node by name, the address chalkctl reaches it at, the image it runs, and the YAML files whose documents form its config, as files every node gets, files of named groups and each node's own. The inventory names each image by its image directory, relative to the inventory's directory; a node runs its own imageName, else that of its groups, else the inventory's spec.all.imageName. Every node that holds a Cluster document holds the same Cluster and Flannel specs. chalkctl reads the file --inventory names, else the file $CHALKOSINVENTORY names, else chalkos.yaml in the current directory. It stays on the operator's machine and names no secrets. Its keys of spec.nodes are the node names; its metadata.name is a label for messages and the default kubeconfig context. See Write a node's config, Config and the inventory and its fields in Config kinds.

Kind

The type of a config document, named by its kind and its apiVersion's group and version. chalkos declares its own kinds in chalkos.dev and its subdomains; an extension declares more under chalkos.kinds in an image's NixOS modules, with a group of its own. An image accepts the kinds its modules declare and refuses any other. A unit reads the documents of the kinds it is granted. See Config and the inventory and Config kinds.

Kubernetes share

The Kubernetes secrets a node receives at install. On a control plane, these are the cluster, front-proxy and etcd CAs with their keys, the service-account key, the encryption key and the node CA; on a worker, the cluster CA certificate and a kubelet client certificate. chalkd keeps it on STATE. See How Kubernetes runs on a node.

Lab

A cluster that chalklab runs as virtual machines, described by a lab file. The lab file, chalklab.yaml, names the cluster's inventory and each VM's MAC address, memory, CPUs and disks. The lab's state (disks, firmware variables, TPM state, keys, logs, kubeconfig, client file and an inventory that reaches the nodes at forwarded ports) lives in $XDG_STATE_HOME/chalklab/<name>, after the lab file's metadata.name. See the quick start and Config kinds.

Layer

A predefined set of NixOS modules that chalkos ships, which an image lists in its modules like any other module to take on a role. chalkos.nixosModules.kubernetes enables chalkos's Kubernetes modules, and chalkos.nixosModules.router its router modules. An image lists one layer at most, and an image without one runs chalkos's base alone. A layer marks the image in /etc/chalkos/layers, and chalkd and chalkctl run the layer's plugin for it. See mkImages.

Maintenance mode

The mode chalkd runs in on a node that is not installed. In it, chalkd serves a self-signed certificate, shows its fingerprint and the node's addresses on the console, and accepts only reads, diagnostics, a reboot, a shutdown, the enrolment of Secure Boot keys and an install. See Architecture.

Module group

A named set of kernel modules an image carries. Modules declare groups in chalkos.kernel.groups, and an image's groups are listed in chalkos.kernel.moduleGroups. Every image carries the base groups storage, network, virtualisation, filesystems, firewall and platform. An image with Kubernetes adds kubernetes. gpu, sound, media, wireless, infiniband, can and industrial are added in an image's modules. See The image.

Node

A machine of the cluster, named in the inventory. The inventory also names the image it runs. On an image with Kubernetes, its Node document of the group kubernetes.chalkos.dev says whether it is a control plane or a worker, and gives its labels, taints and Kubernetes addresses. It is installed from that image's image directory. See Config and the inventory.

Node CA

An intermediate CA under the OS CA that may issue only server and client certificates. On a Kubernetes cluster, control planes hold it and renew every node's certificate with it, so they never hold the OS CA's key. chalkctl rotate node-ca replaces it. See Certificates.

Node certificate

The certificate chalkd serves and authenticates with. The node CA issues it for the node's name, host name and addresses, valid for one year. chalkd takes the node's name from its subject. Once two thirds of its lifetime have passed, a Kubernetes node renews it through a control plane, and any other node reports that chalkctl node renew is due. See Certificates.

Node name

A node's key in the inventory's spec.nodes, such as cp1. chalkctl puts it into the node certificate at install, and chalkd reads it from there. The hostname defaults to it unless a Host document gives another, and the Kubernetes node name, the etcd member, the seed of the partition UUIDs and the recovery key derive from it, so renaming a node takes a reinstall. See Config and the inventory.

Normal mode

The mode chalkd runs in on an installed node. In it, chalkd serves its node certificate and accepts only clients with a certificate from the OS CA. See Architecture.

OCI image

A chalkos image published to a registry as one OCI artifact. chalkctl reads its header and contract from the registry when the inventory names the image by reference, and nodes pull it in an install from the installer or an upgrade. See Publish images to a registry.

Operator

The client role that may do what a reader may, and also reboot, upgrade, drain and uncordon nodes. An upgrade that delivers config needs an admin. See Security model.

OS CA

The root certificate authority of a cluster's node API: chalkd and chalkctl trust only certificates that chain to it. It lives in the secrets file, is valid for ten years and signs the node CA and client certificates. See Certificates.

Output

A set of config files for a service, rendered from config documents. A program of the image's store renders them, and they are checked before a delivery changes the node, as chalkos.outputs declares it. chalkd renders every output again at boot. See Render config files.

PCR 7

The TPM register that measures the Secure Boot state. It measures whether Secure Boot is on, the PK, KEK, db and dbx contents, and the db certificate that verified the boot binaries. chalkos seals the keys of encrypted partitions to it, so they unseal after an upgrade signed with the same key and not after a change to Secure Boot. See Storage and encryption.

Platform

The kind of machine an image is built for, such as metal or kvm, which each image names in its platform. chalkos has NixOS modules of its own for metal and kvm, which only that platform's images carry, with its drivers and agents. A node refuses an image of another platform than the one it runs. See Images and platforms.

Previous config

The config the running image ran before the last delivery, which the node keeps beside the image's config slot, one level deep. chalkctl config rollback delivers it again, and the config it replaces becomes the previous one. An install or an upgrade writes a slot without one. See Config and the inventory.

Provenance

The git revision and tag an image was built from. chalkos.lib.mkImages takes them as revision and tag and writes them into the image's metadata.json. chalkctl shows them beside the root hash, as in 3fa9c1e2b7d0 (v1.5.0, a1b2c3d4e5f6), and a node keeps them in /state/chalkd/images.json. They never reach the store, and nothing orders or refuses an image by them. See The image.

Reader

The client role that may read node information, disks, status, logs and etcd members. See Security model.

Recovery key

The default fallback keyslot of every encrypted partition, used when the TPM does not unseal. chalkos derives it from the secrets file and the node name, so the file never changes, and chalkctl recovery-key prints it. See Storage and encryption.

Revoked node

A node shut out of the cluster by name with chalkctl node revoke. Control planes list it in the ConfigMap kube-system/chalkos-revoked-nodes and refuse to renew its node certificate, and the API server denies every write by its kubelet. Its certificates stay valid until they expire. chalkctl node unrevoke lets the name back in. See Kubernetes certificates.

Rollback

The return to the previous image after a new one used up its boot tries without a blessed boot. chalkd reports it in status with the failed boot's last log lines, and chalkctl upgrade stops. Two more rollbacks run on request: chalkctl upgrade --rollback boots the image the inactive slot still holds, and chalkctl config rollback delivers the previous config again, as the node does by itself at the end of an unconfirmed trial. See Boot, health and rollback.

Root hash

The top hash of the store's dm-verity hash tree. The UKI carries it on the kernel command line (usrhash=), so the UKI's signature covers every byte of the store. It identifies an image: chalkd and chalkctl tell images apart by it alone, and chalkctl shows its first 12 hex characters. See The image.

Rotation

The replacement of a CA or key, in phases, with chalkctl rotate. The phases first trust the new value everywhere, then use it, then re-issue, and last remove the old value. Rotation also revokes what the old value issued. See Certificates.

Secrets file

The file that holds every secret of a cluster, written once by chalkctl gen secrets. It holds the OS CA and node CA, the Kubernetes CAs and keys, and the secret recovery keys derive from. It is secrets.age, encrypted with age, or secrets.json; its public part is secrets.pub.json. chalkctl reads it only where --secrets names it, or from standard input with --secrets -; an inventory names none. See Set up your workstation and Security model.

Secure Boot

The UEFI firmware's check that a boot binary is signed by a certificate in db. chalkos signs systemd-boot and the UKI, whose signature covers the store through the root hash. See Security model.

Slot

A pair of partitions that holds one image's store: the erofs data and its dm-verity hash tree. They are labelled store_<hash16> and store-verity_<hash16> from the first 16 hex characters of its root hash, or _empty. A node has two, A and B. See The image.

STATE

A small partition that holds a node's config, certificates and keys. It is 128 MiB, encrypted and sealed to the TPM, all by default. It holds the node's config, node certificate, trusted CAs, Kubernetes share and storage definitions, and the marker that the node is installed. See Storage and encryption.

Store

The read-only Nix store of an image: an erofs file system compressed with zstd, checked by dm-verity on every read. See The image.

TPM

The machine's Trusted Platform Module 2.0, which seals disk keys to the Secure Boot state. It seals the keys of STATE, VAR and encrypted volumes to PCR 7, so they unlock without a password only under the same Secure Boot state. See Security model.

Trial

A config delivered with chalkctl config apply --try, which the node rolls back to its previous config at a deadline unless chalkctl confirms it. While it is open, the node refuses other deliveries, upgrades and image rollbacks, and a reboot ends it with the config from before it. See Config and the inventory.

UKI

A unified kernel image: the kernel, initrd, kernel command line and os-release in one EFI binary. systemd-boot loads it and Secure Boot verifies it. chalkos keeps one per slot on the ESP as chalkos_<generation>.efi. See The image.

Upgrade

The installation of another image into a node's inactive slot, from which it boots next. That boot uses boot counting. chalkctl upgrade rolls it through a cluster. A Kubernetes upgrade is an image upgrade. See Upgrades.

Validator

A program of the image's store that checks a node's config documents for what the kinds' schemas and change rules do not see, and reports the refusals it finds. chalkd runs every validator before a delivery changes the node, as chalkos.validators declares it. See Validate config with a program.

VAR

The partition mounted at /var that holds what a node writes. That includes etcd, containerd, the kubelet and logs. The node's SystemDisk document sizes it; by default it fills the rest of the system disk, and it is encrypted like STATE. See Storage and encryption.

VIP

A virtual IP address of the API server, one per IP family, that one healthy control plane holds. chalkd elects the holder through etcd and announces the address on the local network. See Kubernetes networking.

Volume

A partition beyond VAR, declared by a Volume document in a node's config. It sits on the system disk or a disk of its own, encrypted or not, formatted and mounted or left raw. See Storage and encryption.

Worker

A node that runs containerd and the kubelet but no control plane components. Its Node document sets spec.type: worker. Its kubelet joins the cluster at the cluster endpoint with the certificate in its Kubernetes share. See How Kubernetes runs on a node.