Glossary¶
Each entry defines a term the documentation uses repeatedly and links to the page that explains it. The first sentence of an entry is a short definition, which every page shows as a tooltip on the term. Option, command and API names link to the generated reference.
A/B¶
The update scheme of a node's disk, with two slots, A and B, that each hold an image. The node boots one slot while an upgrade writes the other, so the image it ran before stays on disk for a rollback. See The image.
Admin¶
The client role that may call almost every method of the node API. It may call
every method but the one nodes renew their certificates with, which needs the role node:
install, config changes, volume resets, bootstrap, etcd membership changes and rotations among
them. See Security model.
Blessed boot¶
A boot that the health check found healthy. After it,
systemd-bless-boot removes the boot counter from the UKI's file name. Nothing rolls a
blessed image back on its own. See Boot, health and rollback.
Boot counting¶
systemd-boot's count of the tries a newly installed image has left. The count is kept in its
UKI's file name (chalkos_<generation>+3.efi). Each boot that is not blessed
uses a try; with none left, systemd-boot boots the previous image. The number of tries is
chalkos.upgrade.bootTries. See
Boot, health and rollback.
Bootstrap¶
The one-time start of a cluster's etcd on its first control plane. It runs
with chalkctl bootstrap and is refused on a node that is bootstrapped or holds etcd data, so a cluster is never initialised
twice; further control planes join on their own. See
The control plane.
chalkbuild¶
The command that prepares the images Nix builds for a cluster's nodes. chalkbuild sign signs
a copy of an image directory with the Secure Boot db key, on
the machine that holds the key, and checks the copy against the db certificate. It never talks
to nodes. See its commands and
Sign images for Secure Boot.
chalkcfg¶
The command on every node that prints the node's config documents. It reads them
from /run/chalkos/config and prints them by kind and name, as names, JSON, YAML, a jsonpath
expression or a Go template. It has no network access and never prints Secrets or the documents of
withheld kinds, such as Addons. An image's NixOS modules build its commands with
chalkos.kinds.<Kind>.get, which checks a path against the
kind's options when the image is evaluated. See its commands.
chalkctl¶
The command-line client of chalkos. It reads a cluster from an inventory and the image directories it names for the nodes, installs, upgrades and operates nodes through chalkd, and issues certificates from the secrets file. It never builds or signs images or runs Nix. See its commands and Architecture.
chalkd¶
The agent on every node. It serves the node API on TCP port 50000 over mutual TLS, installs and upgrades the node, applies its config, runs the Kubernetes control plane and decides whether a boot is healthy. See Architecture.
chalklab¶
The command that runs a cluster's kvm nodes as QEMU virtual machines on one machine. Each
virtual machine has Secure Boot and a TPM, and chalklab installs and bootstraps them. See
its commands and the quick start.
Change rule¶
A CEL expression that a config must make true for the image to accept it. An image's schemas
declare it on a kind or on one of its fields. A rule that is false refuses the config,
or says it takes chalkctl storage reset or a reinstall. See
Guard changes with rules and facts.
Client file¶
A JSON file that lets someone operate nodes without the secrets file. It holds a
client certificate of one client role, its key and the OS CA
certificate, and chalkctl client new issues it. It names no cluster: its OS CA ties it to one,
and a node of another cluster fails the TLS handshake with it. See
Set up your workstation and
Security model.
Client role¶
The permission a client certificate grants on the node API: reader,
operator or admin. The role is taken from the certificate's Organization,
and each role is allowed what the roles before it are. A certificate issued through the
node CA has the role node, which may only renew its node certificate. See
Security model.
Cluster endpoint¶
The URL of the cluster's API server, which kubelets, kubeconfigs and joining control planes use.
It is spec.endpoint of the Cluster document. Nodes also renew their
node certificates at port 50000 of its host, so it points at a control plane,
a VIP or a load balancer that forwards both ports. See
Kubernetes networking.
Cluster registry¶
A container registry on a Kubernetes cluster's control planes, which containerd on every node asks
first for the images of the hosts it mirrors. The ClusterRegistry
document lists its endpoints, and chalkctl registry push fills each of them, since they do not
copy images to each other. See
Run an air-gapped cluster with the cluster registry.
Config¶
A node's configuration at runtime: a set of config documents. Each document
is of a kind its image accepts, such as Node,
Network, SystemDisk, Cluster or an extension's own. The inventory names the
YAML files whose documents form a node's set. chalkctl checks the set against the schemas of the
node's image and delivers it at install and with
chalkctl config apply; chalkd checks it again, keeps it on
STATE and applies it at every boot. A new set replaces the node's previous one as a
whole. Secrets travel as Secret documents and reach only the services that read the kinds
referencing them. See Config and the inventory.
Config document¶
One object of a node's config, written in YAML like a Kubernetes object. It has an apiVersion of a group and a version, a kind, a metadata.name and a
spec. The schema of its kind defines the spec; Config kinds lists the
fields of the built-in kinds. On the node, chalkd writes each document except Secrets and
those of withheld kinds, such as Addons, to /run/chalkos/config/<group>/<Kind>/<name>.json. See
Config and the inventory.
Config slot¶
The directory on STATE that holds a node's config for one image, named by the image's root hash.
It is /state/config/<root-hash> on STATE, and holds the documents as delivered, the
same documents with the schemas' defaults filled in, the config's version and the node's
identity. Each image's boot loads its own slot, so a rollback boots the
previous image with the config it ran. An install or an upgrade writes the slot of the
image it writes, and chalkctl config apply that of the running image. An upgrade removes the
slots of images no UKI on the ESP boots any more, so a node holds at most two. See
Config and the inventory.
Contract¶
What an image accepts as config: its name, platform, kind schemas and what it fixes
about Kubernetes. The name and platform are the ones the image was built with, and the schemas
are the JSON Schemas of its kinds. The image carries the schemas in
/etc/chalkos/schemas and the Kubernetes facts in /etc/chalkos/kubernetes/image.json, and its
image directory has them as schemas/ and facts/kubernetes.json. chalkctl
reads the contract from the image directory when it validates config, installs from the
installer or upgrades a node, and from the running node with the
Contract method when it applies config. On an upgrade, the node reads
it from the image it wrote. See Image and partition layout.
Control plane¶
A node that runs etcd and the Kubernetes API server, controller-manager and scheduler. Its Node
document sets spec.type: controlplane. It runs those components as static pods that chalkd
renders, besides containerd and the kubelet, and it is the only kind of node chalkctl delivers
Addon documents to. See The control plane.
db and dbx¶
The UEFI firmware's Secure Boot databases: db lists what the firmware boots, dbx what it refuses. Both hold certificates and hashes of binaries. A chalkos image boots under Secure Boot when its signer's certificate is in db and nothing of it is in dbx. See The image.
Diagnostic¶
A read-only command of an image that chalkd runs on a node for a client, such as ip route get.
Its image declares it in chalkos.diagnostics, with typed
parameters and the least client role that runs it, and chalkd runs it in a
transient systemd unit with a sandbox of its own. See
chalkctl node diag.
dm-verity¶
The kernel's block-level integrity check, which makes a modified block of the store fail to read. Each block of the store is hashed into a hash tree whose top is the root hash, and a block that does not match its hash fails to read. See The image.
Effect¶
An action on the node's services after a delivery is applied, such as a unit restart. An effect
restarts or reloads units, runs a command or runs a layer's action in chalkd, as
chalkos.effects declares it. It fires when one of its
outputs or the documents of one of its kinds changed, or when its CEL rule, like a
change rule, holds. See
React to changes.
ESP¶
The EFI system partition, a FAT file system at the start of the system disk. It holds systemd-boot and the UKIs of both slots, and is 1 GiB by default. See Image and partition layout.
etcd member¶
A control plane that belongs to the cluster's etcd. It is a learner while it catches up and a
voter after. A control plane pins its addresses on STATE when it bootstraps, or before
it adds itself as a learner, and every later boot waits for exactly them;
chalkctl etcd leave undoes both. See The control
plane.
Extension¶
NixOS modules outside chalkos that add services to an image and may declare config
kinds. An extension sits in an image's modules and declares its kinds under
chalkos.kinds, whose documents a node's YAML files then carry. The
services read those documents through a grant or as outputs rendered from
them, and effects restart them when they change. chalkos's own Kubernetes support
declares its kinds and effects the same way. See
Declare a config kind.
Fact¶
Node state that a change rule reads. Facts are declared in
chalkos.facts: the JSON of a file, such as the
Kubernetes share, or of a program of the image's store, which chalkd runs in
a sandbox. See
Guard changes with rules and facts.
Fallback keyslot¶
The second keyslot of every encrypted partition, which unlocks it when the TPM does not.
It holds the node's recovery key by default, or a password, or the partition has
none, as spec.encryption.fallback of the node's SystemDisk chooses.
See
Storage and encryption.
Generation¶
The number in a UKI's file name that orders a node's boot entries, the highest first. A node
counts it itself: the file name of a UKI on the ESP is chalkos_<generation>.efi.
The image and an install write
generation 1; each upgrade writes the one above the highest on the ESP, so
systemd-boot, which orders a node's UKIs by file name, boots the image installed last first,
older or newer. See
Boot and the image.
Grant¶
The config kinds a unit may read at runtime. A unit's grant is declared in
systemd.services.<unit>.chalkos.kinds: chalkd writes
the documents of those kinds below /run/chalkos/config and the values their
secretRef fields name into the unit's credentials. A grant restarts nothing; an
effect does. See
Read config in a service.
Health check¶
The check that decides whether a counted boot is healthy. chalkos-health.service runs it. chalkd must serve on every node, and on a node with Kubernetes the preparation must
have succeeded. Then a control plane that is an etcd member needs its local etcd member, its API
server and its kubelet healthy, and a worker its kubelet healthy and its Node registered. A node
without Kubernetes, or a control plane not bootstrapped yet, needs its config applied and no unit
of the boot failed. A boot that passes within
chalkos.upgrade.healthTimeout is
blessed. See Boot, health and rollback.
Identity¶
The secret-free form that chalkd translates a node's config into before applying it.
It holds the node's hostname, static addresses, labels, taints, storage and Kubernetes
addresses. chalkd keeps it on STATE in each image's config slot, translated with
that image's schemas, and the image's boot applies it. chalkctl validate translates a set the
same way to check it. See Image and partition layout.
Image¶
The whole operating system of a node as one unsigned raw disk image. It has the ESP and slot A
filled and is built by chalkos.lib.mkImage, or chalkos.lib.mkImages for a list of images, from a
name, a platform and NixOS modules, a layer among them. Its name is
CHALKOS_IMAGE in its os-release and name in its metadata.json, and chalkctl upgrade --image
gives it to the nodes that run an image of that name. Every node that runs it boots the same bytes;
what differs between nodes travels in their config. Its build output is an image
directory. See The image.
Image directory¶
The build output of an image: the raw disk image with its metadata and config schemas.
It is what chalkos.lib.mkImage returns, such as images.node-metal of chalkos.lib.mkImages,
and holds the raw image, systemd-repart's report of its partitions, the definitions of the system
region, metadata.json, which names the image's name, platform, root hash,
provenance and kinds, the kinds' JSON Schemas in schemas/ and the facts of its
layers in facts/. An image with Kubernetes has facts/kubernetes.json, which names the release
of its kubelet and its pod network. The inventory names the image directory in an
image's source.directory, chalkbuild signs a copy of it, and chalkctl reads it to
validate config, install and upgrade.
See Image and partition layout.
Installer¶
An image that always boots into maintenance mode and writes a node's image to a
disk of the machine. chalkos.lib.mkImages builds one, as a raw image and a hybrid ISO, with
installer.modules; it installs images of every platform. See
Architecture.
Inventory¶
The file that tells chalkctl about a cluster: its nodes, their addresses, images and config
files. It lists each node by name, the address chalkctl reaches it at, the image it
runs, and the YAML files whose documents form its config, as files every node gets,
files of named groups and each node's own. The inventory names each image by its
image directory, relative to the inventory's directory; a node runs its own
imageName, else that of its groups, else the inventory's spec.all.imageName. Every node that
holds a Cluster document holds the same Cluster and Flannel specs. chalkctl reads the file
--inventory names, else the file $CHALKOSINVENTORY names, else chalkos.yaml in the current
directory. It stays on the operator's machine and names no secrets. Its keys of spec.nodes are the node names;
its metadata.name is a label for messages and the default kubeconfig context. See
Write a node's config,
Config and the inventory
and its fields in Config kinds.
Kind¶
The type of a config document, named by its kind and its apiVersion's
group and version. chalkos declares its own kinds in chalkos.dev and its subdomains; an extension declares more under
chalkos.kinds in an image's NixOS modules, with a group of its own.
An image accepts the kinds its modules declare and refuses any other. A unit reads the documents
of the kinds it is granted. See
Config and the inventory and
Config kinds.
Kubernetes share¶
The Kubernetes secrets a node receives at install. On a control plane, these are the cluster, front-proxy and etcd CAs with their keys, the service-account key, the encryption key and the node CA; on a worker, the cluster CA certificate and a kubelet client certificate. chalkd keeps it on STATE. See How Kubernetes runs on a node.
Lab¶
A cluster that chalklab runs as virtual machines, described by a lab file. The lab
file, chalklab.yaml, names the cluster's inventory and each
VM's MAC address, memory, CPUs and disks. The lab's state (disks, firmware variables, TPM state,
keys, logs, kubeconfig, client file and an inventory that reaches the nodes at forwarded ports)
lives in $XDG_STATE_HOME/chalklab/<name>, after the lab file's metadata.name. See the
quick start and Config kinds.
Layer¶
A predefined set of NixOS modules that chalkos ships, which an image lists in its
modules like any other module to take on a role. chalkos.nixosModules.kubernetes enables
chalkos's Kubernetes modules, and chalkos.nixosModules.router its router modules. An image
lists one layer at most, and an image without one runs chalkos's base alone. A layer marks the
image in /etc/chalkos/layers, and chalkd and chalkctl run the layer's plugin for it. See
mkImages.
Maintenance mode¶
The mode chalkd runs in on a node that is not installed. In it, chalkd serves a self-signed certificate, shows its fingerprint and the node's addresses on the console, and accepts only reads, diagnostics, a reboot, a shutdown, the enrolment of Secure Boot keys and an install. See Architecture.
Module group¶
A named set of kernel modules an image carries. Modules declare groups in
chalkos.kernel.groups, and an image's groups are listed
in chalkos.kernel.moduleGroups. Every image
carries the base groups storage, network, virtualisation, filesystems, firewall and
platform. An image with Kubernetes adds kubernetes. gpu, sound, media, wireless,
infiniband, can and industrial are added in an image's modules. See
The image.
Node¶
A machine of the cluster, named in the inventory. The inventory also names the
image it runs. On an image with Kubernetes, its Node document of the group
kubernetes.chalkos.dev says whether it is a control plane or a
worker, and gives its labels, taints and Kubernetes addresses. It is installed from
that image's image directory. See
Config and the inventory.
Node CA¶
An intermediate CA under the OS CA that may issue only server and client certificates.
On a Kubernetes cluster, control planes hold it and renew every node's certificate with it, so
they never hold the OS CA's key. chalkctl rotate node-ca replaces it. See
Certificates.
Node certificate¶
The certificate chalkd serves and authenticates with. The node CA issues it for the
node's name, host name and addresses, valid for one year. chalkd takes the node's name from its
subject. Once two thirds of its lifetime have passed, a Kubernetes node renews it through a control
plane, and any other node reports that chalkctl node renew is due. See
Certificates.
Node name¶
A node's key in the inventory's spec.nodes, such as cp1. chalkctl puts it into the node certificate at
install, and chalkd reads it from there. The hostname defaults to it unless a Host document
gives another, and the Kubernetes node name, the etcd member, the seed of the partition UUIDs and
the recovery key derive from it, so renaming a node takes a reinstall. See
Config and the inventory.
Normal mode¶
The mode chalkd runs in on an installed node. In it, chalkd serves its node certificate and accepts only clients with a certificate from the OS CA. See Architecture.
OCI image¶
A chalkos image published to a registry as one OCI artifact. chalkctl reads its header and contract from the registry when the inventory names the image by reference, and nodes pull it in an install from the installer or an upgrade. See Publish images to a registry.
Operator¶
The client role that may do what a reader may, and also reboot, upgrade, drain and uncordon nodes. An upgrade that delivers config needs an admin. See Security model.
OS CA¶
The root certificate authority of a cluster's node API: chalkd and chalkctl trust only certificates that chain to it. It lives in the secrets file, is valid for ten years and signs the node CA and client certificates. See Certificates.
Output¶
A set of config files for a service, rendered from config documents. A program of the image's
store renders them, and they are checked before a delivery changes the node, as
chalkos.outputs declares it. chalkd renders every output again at
boot. See
Render config files.
PCR 7¶
The TPM register that measures the Secure Boot state. It measures whether Secure Boot is on, the PK, KEK, db and dbx contents, and the db certificate that verified the boot binaries. chalkos seals the keys of encrypted partitions to it, so they unseal after an upgrade signed with the same key and not after a change to Secure Boot. See Storage and encryption.
Platform¶
The kind of machine an image is built for, such as metal or kvm, which each image
names in its platform. chalkos has NixOS modules of its own for
metal and kvm, which only that platform's images carry, with its drivers and agents. A node
refuses an image of another platform than the one it runs. See Images and platforms.
Previous config¶
The config the running image ran before the last delivery, which the node keeps beside
the image's config slot, one level deep.
chalkctl config rollback delivers it again, and the
config it replaces becomes the previous one. An install or an upgrade writes a slot without one.
See Config and the inventory.
Provenance¶
The git revision and tag an image was built from. chalkos.lib.mkImages takes them as revision
and tag and writes them into the image's metadata.json. chalkctl shows them beside
the root hash, as in 3fa9c1e2b7d0 (v1.5.0, a1b2c3d4e5f6), and a node keeps them in
/state/chalkd/images.json. They never reach the store, and nothing orders or refuses an image by
them. See The image.
Reader¶
The client role that may read node information, disks, status, logs and etcd members. See Security model.
Recovery key¶
The default fallback keyslot of every encrypted partition, used when the TPM
does not unseal. chalkos derives it from the
secrets file and the node name, so the file never changes, and
chalkctl recovery-key prints it. See
Storage and encryption.
Revoked node¶
A node shut out of the cluster by name with chalkctl node revoke.
Control planes list it in the ConfigMap kube-system/chalkos-revoked-nodes and refuse to renew
its node certificate, and the API server denies every write by its kubelet. Its certificates stay valid
until they expire. chalkctl node unrevoke lets the name back in. See
Kubernetes certificates.
Rollback¶
The return to the previous image after a new one used up its boot tries without a
blessed boot. chalkd reports it in status
with the failed boot's last log lines, and chalkctl upgrade stops. Two more rollbacks run on
request: chalkctl upgrade --rollback boots the image the inactive
slot still holds, and chalkctl config rollback
delivers the previous config again, as the node does by itself at the end of an
unconfirmed trial. See
Boot, health and rollback.
Root hash¶
The top hash of the store's dm-verity hash tree. The UKI carries it on the kernel command line (usrhash=), so the UKI's signature covers
every byte of the store. It identifies an image: chalkd and chalkctl tell images apart by it
alone, and chalkctl shows its first 12 hex characters. See
The image.
Rotation¶
The replacement of a CA or key, in phases, with chalkctl rotate. The
phases first trust the new value everywhere, then use it,
then re-issue, and last remove the old value. Rotation also revokes what the old value issued.
See
Certificates.
Secrets file¶
The file that holds every secret of a cluster, written once by
chalkctl gen secrets. It holds the OS CA and node CA, the
Kubernetes CAs and keys, and the secret recovery keys derive from. It is secrets.age, encrypted
with age, or secrets.json; its public part is secrets.pub.json. chalkctl reads it only where
--secrets names it, or from standard input with --secrets -; an inventory names none. See
Set up your workstation and
Security model.
Secure Boot¶
The UEFI firmware's check that a boot binary is signed by a certificate in db. chalkos signs systemd-boot and the UKI, whose signature covers the store through the root hash. See Security model.
Slot¶
A pair of partitions that holds one image's store: the erofs data and its dm-verity hash
tree. They are labelled store_<hash16> and store-verity_<hash16>
from the first 16 hex characters of its root hash, or _empty. A node has two, A
and B. See The image.
STATE¶
A small partition that holds a node's config, certificates and keys. It is 128 MiB, encrypted and sealed to the TPM, all by default. It holds the node's config, node certificate, trusted CAs, Kubernetes share and storage definitions, and the marker that the node is installed. See Storage and encryption.
Store¶
The read-only Nix store of an image: an erofs file system compressed with zstd, checked by dm-verity on every read. See The image.
TPM¶
The machine's Trusted Platform Module 2.0, which seals disk keys to the Secure Boot state. It seals the keys of STATE, VAR and encrypted volumes to PCR 7, so they unlock without a password only under the same Secure Boot state. See Security model.
Trial¶
A config delivered with chalkctl config apply --try, which the node rolls back to its
previous config at a deadline unless chalkctl confirms it. While it is open, the
node refuses other deliveries, upgrades and image rollbacks, and a reboot ends it with the config
from before it. See
Config and the inventory.
UKI¶
A unified kernel image: the kernel, initrd, kernel command line and os-release in one EFI
binary. systemd-boot loads it and Secure Boot verifies it. chalkos keeps one per slot on the ESP
as chalkos_<generation>.efi. See The image.
Upgrade¶
The installation of another image into a node's inactive slot, from which it boots next.
That boot uses boot counting.
chalkctl upgrade rolls it through a cluster. A Kubernetes upgrade is
an image upgrade. See Upgrades.
Validator¶
A program of the image's store that checks a node's config documents for what the kinds' schemas
and change rules do not see, and reports the refusals it finds. chalkd runs every validator before
a delivery changes the node, as chalkos.validators
declares it. See Validate config with a program.
VAR¶
The partition mounted at /var that holds what a node writes. That includes etcd, containerd,
the kubelet and logs. The node's SystemDisk document sizes it; by
default it fills the rest of the system disk, and it is encrypted like STATE. See
Storage and encryption.
VIP¶
A virtual IP address of the API server, one per IP family, that one healthy control plane holds. chalkd elects the holder through etcd and announces the address on the local network. See Kubernetes networking.
Volume¶
A partition beyond VAR, declared by a Volume document in a node's
config. It sits on the system disk or a disk of its own,
encrypted or not, formatted and mounted or left raw. See
Storage and encryption.
Worker¶
A node that runs containerd and the kubelet but no control plane components. Its Node document
sets spec.type: worker. Its kubelet joins the cluster at the
cluster endpoint with the certificate in its
Kubernetes share. See
How Kubernetes runs on a node.