Image and partition layout¶
This page is for operators and image authors who read a node's disks, its ESP or STATE, or the outputs of an image build. It lists the partitions of a node's disks, the files on the ESP and on STATE, the outputs of an image build and the paths a running node uses. The image and Storage and encryption explain why the layout is what it is.
Partition map¶
The system disk holds the system region (the ESP, two slots and STATE) followed by VAR and the volumes placed on it. A slot is a pair of partitions: the dm-verity hash tree first, then the erofs data of the store.
flowchart LR
ESP["ESP<br/>vfat, 1 GiB"]
subgraph A["slot A"]
VA["store-verity_hash16<br/>128 MiB"]
SA["store_hash16<br/>3 GiB"]
end
subgraph B["slot B"]
VB["_empty<br/>128 MiB"]
SB["_empty<br/>3 GiB"]
end
STATE["state<br/>128 MiB"]
VAR["var<br/>rest of the disk"]
VOL["volumes<br/>60-name"]
ESP --> VA --> SA --> VB --> SB --> STATE --> VAR --> VOL
The table lists the partitions in disk order. The first column is the systemd-repart
definition file that describes each one. The system region's sizes are the defaults of the
image's chalkos.disk options, which Image options describes. They are
fixed when the image is built, so every node that runs one image has the same
system region.
| Definition | Label | Size | File system | Encryption | Created by |
|---|---|---|---|---|---|
00-esp |
esp |
espSize, 1G |
vfat | none | image build |
10-store-verity-a |
store-verity_<hash16> |
storeVeritySize, 128M |
dm-verity hash tree | none | image build |
20-store-a |
store_<hash16> |
storeSize, 3G |
erofs, zstd | none | image build |
30-store-verity-b |
_empty |
storeVeritySize, 128M |
none until an upgrade | none | first boot, or install from the installer |
40-store-b |
_empty |
storeSize, 3G |
none until an upgrade | none | first boot, or install from the installer |
50-state |
state |
stateSize, 128M |
ext4 | node's policy, LUKS2 sealed to the TPM by default | first boot, install |
50-var |
var |
the SystemDisk's spec.var.size, rest of the disk |
ext4 | node's policy | install |
60-<name> |
<name> |
the volume's size |
the volume's format |
node's policy | install, later config |
A volume with a disk of its own is the only partition on that disk, from the definition
10-<name>. The system region's definitions are the same files the image ships in
/etc/chalkos/repart.d; VAR's and the volumes' come from the node's
config and are kept on STATE.
The "Created by" column names the first writer of each partition:
- Image build: the Nix build of the image runs systemd-repart and writes the ESP and slot A into the raw image.
- First boot: systemd-repart in the initrd runs the system region's definitions against the disk systemd-boot was loaded from, so it adds slot B and STATE behind the partitions the image shipped with. It runs at every boot and changes nothing once they exist.
- Install: from the installer, chalkd lays out the
whole system region on the target disk with one systemd-repart run of the image's definitions and
writes slot A itself. In place, chalkd recreates STATE when its contents do not match the node's
encryption policy (an unencrypted STATE for a
SystemDiskwithencryption: {mode: none}). In both flows it then creates VAR and the volumes. - Config:
chalkctl config applycreates volumes added to a node's storage and grows those whose size grew. At every boot,chalkos-storagein the initrd applies the storage section recorded on STATE again.
The sizes have lower bounds, checked when the image is evaluated: the ESP needs at least 260
MiB, the smallest ESP systemd-repart formats as vfat, and STATE at least 64 MiB, the smallest
encrypted ext4 partition it makes. The image build fails when the store's data takes more than
80% of a slot, its hash tree more than 80% of the verity partition, or three UKIs more than 80%
of the ESP. The installer sets storeSize and storeVeritySize to
null, which sizes its single slot to its contents, and its ESP to 260 MiB.
GPT partition types¶
systemd-repart matches existing partitions to definitions by GPT type, so STATE, VAR and each volume have a type of their own, a hash of the label, and no definition takes over another one's partition.
| Partition | GPT type |
|---|---|
| ESP | c12a7328-f81f-11d2-ba4b-00a0c93ec93b |
| Store hash tree, x86-64 | 77ff5f63-e7b6-4633-acf4-1565b864c0e6 |
| Store data, x86-64 | 8484680c-9521-48c6-9c11-b0720656f69e |
| Store hash tree, arm64 | 6e11a4e7-fbca-4ded-b9e9-e1a512bb664e |
| Store data, arm64 | b0e01050-ee5f-4390-949a-9101b17104e9 |
| STATE | 480b7842-1236-1abb-0339-e5503b43122a |
| VAR | 65f335d7-a1f7-f6df-b954-a97d9a5db9e6 |
Volume <name> |
the first 128 bits of SHA-256 of chalkos-partition-type:<name>, as a UUID |
The store's types are the Discoverable Partitions Specification's usr types. STATE, VAR and
volumes have chalkos's own types.
Partition UUIDs¶
The UKI's kernel command line carries the store's
root hash as usrhash=, and systemd finds the store's partitions by
UUIDs derived from it: the data partition's UUID is the root hash's first 128 bits, the hash
tree's its last 128 bits. A store
built with the root hash 2ece03b0fa094b775ee5bf2e1f6f9eb66d6f0dc06f30f49f2f978f00e1173942
therefore lives in these partitions:
| Partition | Label | Partition UUID |
|---|---|---|
| Store data | store_2ece03b0fa094b77 |
2ece03b0-fa09-4b77-5ee5-bf2e1f6f9eb6 |
| Store hash tree | store-verity_2ece03b0fa094b77 |
6d6f0dc0-6f30-f49f-2f97-8f00e1173942 |
The other UUIDs are set as follows:
| Partition | Partition UUID |
|---|---|
| A slot being written | random, with both labels _empty, so nothing boots it |
| A written slot | derived from the root hash, with the labels naming its first 16 hex characters |
| ESP | random, set at install, because systemd-boot reports it as the boot partition and no other disk with the same image may match it |
| STATE | random, from systemd-repart's --seed=random, when an install from the installer lays out the disk or an install recreates STATE for another encryption; otherwise the one systemd-repart gave it at first boot |
| VAR and volumes | derived by systemd-repart from a seed per node and disk name |
The labels name a slot's store by the first 16 hex characters of its root hash, <hash16>, so
store-verity_<hash16> fits a GPT label of 36 characters, but nothing boots by label; the boot
finds the store by partition UUID alone. systemd-repart labels slot A store-verity and store
when it builds the raw image, before the root hash is known; the image build then relabels the
copy in the image directory, so chalkos.raw and its repart-output.json carry the hash's
labels. The installer's own medium keeps store-verity and store.
Files on the ESP¶
The ESP is mounted at /efi when it is used, with mode 0700, and unmounted after a minute
idle, so a power cut rarely finds its FAT file system in use.
| Path | What it is |
|---|---|
/EFI/BOOT/BOOTX64.EFI |
systemd-boot, at the removable-media path, so firmware finds it without a boot entry. BOOTAA64.EFI on arm64. |
/EFI/Linux/chalkos_<generation>.efi |
A UKI without a boot counter: chalkos_1.efi as the image holds it and an install from the installer writes it, or one whose boot was blessed. |
/EFI/Linux/chalkos_<generation>+<left>.efi |
A UKI an upgrade installed, one generation above the highest on the ESP, with chalkos.upgrade.bootTries tries left, such as chalkos_2+3.efi. |
/EFI/Linux/chalkos_<generation>+<left>-<done>.efi |
The same UKI after systemd-boot counted a boot that was not blessed, such as chalkos_2+2-1.efi. +0-3 means no tries left: systemd-boot boots another entry. |
/EFI/Linux/.chalkos-<root-hash>.efi |
A UKI an upgrade or an install from the installer is receiving, named by the image's full root hash. systemd-boot ignores names starting with a dot. |
/EFI/BOOT/.boot-loader.tmp |
The boot loader an install from the installer is writing. |
chalkos is the image ID, system.image.id; the installer's UKI is
chalkos-installer_1.efi. The image ID consists of lower-case letters, digits, ., _ and
-; the build fails otherwise. The generation is the number in the name without its counter; an
upgrade reads the highest any name chalkos_<n>.efi on the ESP gives and writes the next. The
name alone counts, as systemd-boot orders the entry by it: a file that is no UKI, or a UKI whose
os-release names another image or none, counts too. A name that gives no generation, such as
chalkos.efi, counts for none. An upgrade refuses to run while the entry the node booted, or
another it keeps, gives no generation, a generation of 0 or one with leading zeros, such as
chalkos.efi, chalkos_5a.efi, chalkos_7+2147483648.efi, whose counter systemd-boot ignores,
or chalkos_01.efi, or while a name gives a generation too large for one to follow it in 64
bits. chalkd never writes such names.
An upgrade first removes every UKI of the image's ID, and every file that is no UKI whose name
gives a generation, except the one the node booted and those booting the running store, then
receives the new one, so the ESP holds the UKIs of both slots and, during an upgrade, a third. An
install from the installer removes every UKI on the ESP but the chalkos_1.efi it writes, of
any image ID or of none: the installer laid the ESP out, and an earlier attempt with another
image may have left that image's UKI. systemd-boot sorts entries by image ID, lowest first,
before their file names, so a UKI whose image ID sorts before chalkos, such as one with
IMAGE_ID=acme, would boot before every chalkos entry. chalkos itself never puts UKIs of other
image IDs on the ESP. An upgrade leaves systemd-boot alone: only an
install writes the boot loader.
systemd-boot boots the highest generation whose tries are not used up, comparing the numbers as
numbers, by the rules Boot and the image lists. chalkd sets no EFI variable to choose an entry. An install from the installer adds a UEFI
boot entry labelled chalkos for the target ESP's removable-media path, \EFI\BOOT\BOOTX64.EFI
on x86-64 and \EFI\BOOT\BOOTAA64.EFI on arm64.
Files on STATE¶
STATE is mounted at /state. It holds what makes the node this node and nothing else: a config
slot for each image the node holds, below config/<root-hash>, and the machine state every image
shares. The chalkd and kubernetes directories, and each slot's secrets directories, are
readable by root alone.
| Path | Written by | What it holds |
|---|---|---|
config/<root-hash>/ |
install, Upgrade, ApplyConfig | The config slot of the image whose store has the root hash, which its boot loads: a symbolic link to a directory beside it, which a new set replaces as a whole. An Upgrade forgets the slots of the images no UKI on the ESP boots, and with them a slot an upgrade wrote before a crash kept its image from being activated. |
config/<root-hash>/identity.json |
install, Upgrade, ApplyConfig | The node's identity on that image, which its config translates to and the boot applies: hostname, static addresses and storage, and below layers the part of each layer that adds one, such as the Kubernetes node name, addresses, labels and taints. |
config/<root-hash>/delivered/<group>/<Kind>/<name>.json |
install, Upgrade, ApplyConfig | The slot's config documents as delivered, which GetConfig returns with secrets hashed; Secret documents in delivered/secrets/<name>.json. |
config/<root-hash>/documents/<group>/<Kind>/<name>.json |
install, Upgrade, ApplyConfig | The same documents with their schemas' defaults filled in, which the node's services read; Secret documents in documents/secrets/<name>.json. |
config/<root-hash>/version |
install, Upgrade, ApplyConfig | The SHA-256 of the slot's config as delivered, which Status reports. |
chalkd/node.pem |
install, renewal | The node certificate's chain and its key in one file, so both are replaced together. |
chalkd/ca.crt |
install, rotation | The certificates of the OS CAs the node trusts, a bundle. |
kubernetes/share.json |
install, rotation | The node's Kubernetes share. |
kubernetes/node-ip |
chalkd | A control plane's addresses, pinned when it became an etcd member, one per line. |
kubernetes/joining |
chalkd | The node started joining the cluster, so it never bootstraps a cluster of its own. |
kubernetes/bootstrapped |
Bootstrap | The node bootstrapped the cluster. |
kubernetes/etcd-initialised |
chalkd | etcd answered ready after the bootstrap, so its data exists. |
kubernetes/etcd-initial-cluster |
chalkd | etcd's initial cluster when the node joined an existing one. |
kubernetes/left |
EtcdLeave | The node left etcd; it joins again only once reinstalled. |
storage/storage.json |
install, Upgrade, ApplyConfig | The storage as applied, which every image's boot sets up: machine state, which the storage kinds' change rules compare the config of every slot write with. Every boot sets up each volume the file records, also one the booted image's config lacks, and nothing at boot removes a volume. |
storage/disks.json |
chalkos-storage, chalkd | The disks the section's names were pinned to, and the PARTUUID of each volume. |
storage/disks/<disk>/*.conf |
chalkos-storage, chalkd | The repart definitions of each disk, from the storage section. |
installed |
install | The marker that the node is installed, written last. |
A node whose STATE has no installed marker starts chalkd in
maintenance mode.
Outputs of an image build¶
An image directory is what mkImage returns, and mkImages returns by the image's
name, such as nix build .#node-kvm in the lab template, whose flake exposes images.node-kvm. It
holds six entries:
| Entry | What it is |
|---|---|
chalkos.raw |
The raw disk image: the ESP, with the UKI as /EFI/Linux/chalkos_1.efi, and slot A labelled with its root hash, with no slot B and no STATE. |
repart-output.json |
systemd-repart's description of the raw image's partitions: type, label, UUID, offset, size and the store's root hash, with slot A's labels as the raw image carries them. chalkbuild sign and chalkctl install find the partitions with it. |
repart.d |
The system region's definitions, 00-esp.conf to 50-state.conf, as the image carries them in /etc/chalkos/repart.d. |
metadata.json |
What the image is: schemaVersion of this file (0), the image ID, the image's name, its platform and the store's root hash (rootHash), the revision and tag mkImages was given, null without them, kinds, the config kinds the image accepts, each with its apiVersion, kind and whether a node may hold several documents of it (multiple), and annotations, those of the image's OCI manifest, such as the Kubernetes version of an image with the Kubernetes layer. chalkctl reads the root hash and the platform from it, and sends the revision and tag with the image when the file names its root hash; chalkctl warns of a file it ignores. |
schemas |
One JSON Schema (draft 2020-12, the schema dialect of OpenAPI 3.1) per kind, named <group>_<version>_<kind>.json, as the image carries them in /etc/chalkos/schemas. |
facts |
The facts of the image's layers, one <layer>.json for each layer that sets chalkos.imageDirectory.facts; empty on an image without such layers. On an image with Kubernetes, facts/kubernetes.json is what the image fixes about Kubernetes, the file the image carries as /etc/chalkos/kubernetes/image.json: the release of its kubelet and its pod network ({"version": "1.37.2", "cni": "flannel"}). An install from the installer sends the facts with schemas, or the installer reads both from the contract layer of an image it pulls, so the installer translates the node's config for the image it writes to lay out the target disk; an upgrade checks each node's config from the inventory against both before it drains a node, and chalkd checks the config it carries against those of the image it wrote. |
Config kinds lists the fields of the kinds in the schemas of an image with the
Kubernetes layer and of a router image.
The definitions of an image with the default sizes read:
[Partition]
Encrypt=tpm2
Format=ext4
Label=state
SizeMaxBytes=128M
SizeMinBytes=128M
Type=480b7842-1236-1abb-0339-e5503b43122a
An install from the installer refuses definitions that lay out more than the system region,
whether chalkctl sends them or the installer reads them from a pulled image's contract. The
definitions must describe six partitions: one ESP formatted as vfat, the store's data and
hash-tree partitions of two slots for the image's architecture, and STATE labelled state and
formatted as ext4. Each file holds one [Partition] section and sets each key at most once, and
only these keys:
| Partition | Keys |
|---|---|
| ESP | Type, Label, Format, SizeMinBytes, SizeMaxBytes |
| Store data and hash tree | Type, Label, SizeMinBytes, SizeMaxBytes |
| STATE | Type, Label, Format, Encrypt, AddValidateFS, SizeMinBytes, SizeMaxBytes |
The chalkos.disk options produce such definitions, so the restriction matters only for an image
whose repart.d was changed by other means.
The installer's directory holds chalkos-installer.raw and its repart-output.json, the hybrid
ISO chalkos-installer.iso, chalkos-installer.iso.json, which describes the ISO's partitions
as repart-output.json does, and a metadata.json of the same fields, with the ID
chalkos-installer, no name or platform, and the installer's root hash, revision and tag.
OCI artifact¶
An OCI artifact holds an image for publishing to a registry. An image of mkImage carries it as
oci, such as images.node-kvm.oci, and chalkbuild oci writes it from
an image directory. Both write an OCI image layout: the file oci-layout, an index.json that
names one manifest, and the blobs below blobs/sha256/. The manifest has the artifact type
application/vnd.chalkos.image.v1, and no index wraps it.
| Part | Media type | Content |
|---|---|---|
| config | application/vnd.chalkos.image.header.v1+json |
The header: image ID, root hash, architecture, platform, revision and tag. |
| layer | application/vnd.chalkos.image.store.v1 |
The store's erofs data. |
| layer | application/vnd.chalkos.image.hash-tree.v1 |
The store's dm-verity hash tree. |
| layer | application/vnd.chalkos.image.uki.v1 |
The UKI. |
| layer | application/vnd.chalkos.image.boot-loader.v1 |
systemd-boot. |
| layer | application/vnd.chalkos.image.contract.v1.tar |
The contract: metadata.json, schemas/, repart.d/ and facts/. |
The header of the image node-metal reads:
{
"imageId": "chalkos",
"rootHash": "2ece03b0fa094b775ee5bf2e1f6f9eb66d6f0dc06f30f49f2f978f00e1173942",
"architecture": "x86-64",
"platform": "metal",
"revision": "28aac64fc0de68df53a376208012ddc5631b2aeb",
"tag": "v1.5.0"
}
architecture is x86-64 or arm64, as systemd names it. revision and tag are those of
metadata.json, empty when the build named none.
The layers are uncompressed, and chalkos finds each one by its media type, not its position. The store layer holds exactly the store data the root hash covers, and the hash tree layer exactly the hash tree's size, so each layer's digest is the hash of the image part an install or upgrade sends.
The contract layer is a tar of metadata.json, schemas/*.json, repart.d/*.conf and
facts/*.json, which an image without layer facts lacks. Its entries are in name order, with the
modification time 0 and owner 0:0, so the same image directory always makes the same layer and
the same manifest. chalkbuild sign replaces the UKI and boot-loader
layers and the manifest, and keeps the other layers and the annotations.
Annotations¶
The manifest's annotations are the annotations of metadata.json, which the image's
chalkos.imageDirectory.annotations sets.
An annotation whose value the build does not know is left out, so an image built without a tag
has no org.opencontainers.image.version.
| Annotation | Set by | Value |
|---|---|---|
org.opencontainers.image.revision |
mkImages's revision |
The git revision the image was built from. |
org.opencontainers.image.version |
mkImages's tag |
The git tag the image was built from, such as v0.1.0. |
org.opencontainers.image.source |
the image's modules | The URL of the image's source; chalkos's release images name https://github.com/trevex/chalkos. |
org.opencontainers.image.created |
the image's modules | When the image was created, in RFC 3339; chalkos's release images name the time of their commit, so a rebuild makes the same manifest. |
dev.chalkos.kubernetes.version |
the Kubernetes layer | The Kubernetes release of the image's kubelet, such as 1.37.2. |
The manifest of a release image reads, in part:
{
"artifactType": "application/vnd.chalkos.image.v1",
"annotations": {
"dev.chalkos.kubernetes.version": "1.37.2",
"org.opencontainers.image.created": "2026-11-19T21:04:11Z",
"org.opencontainers.image.revision": "28aac64fc0de68df53a376208012ddc5631b2aeb",
"org.opencontainers.image.source": "https://github.com/trevex/chalkos",
"org.opencontainers.image.version": "v0.1.0"
}
}
Device links¶
udev rules in the image name the disk systemd-boot was loaded from, so the node never mistakes another disk carrying the same image or the same labels for its own.
| Link | Target |
|---|---|
/dev/disk/chalk-boot-disk |
The boot disk. |
/dev/disk/chalk-boot/<label> |
A partition of the boot disk by its GPT label, such as /dev/disk/chalk-boot/esp, /dev/disk/chalk-boot/state or /dev/disk/chalk-boot/var. |
Paths on a running node¶
| Path | What it holds |
|---|---|
/efi |
The ESP, mounted on use. |
/state |
STATE. |
/var |
VAR; tmpfs until the node is installed. |
/etc/chalkos/repart.d |
The system region's definitions. |
/etc/chalkos/schemas |
The schemas of the config kinds the image accepts; chalkcfg reads their names to know the kinds. |
/etc/chalkos/os-ca.crt |
The OS CA certificate the image carries, when the image is built with the mkImages argument osCA. |
/etc/chalkos/kubernetes/image.json |
What the image fixes about Kubernetes: the release of its kubelet and its pod network, on images with Kubernetes. chalkd builds the node's cluster from it and the node's config. |
/etc/chalkos/layers/<layer> |
An empty marker of each layer the image carries: kubernetes or router. chalkd reads the directory when it starts and serves no method of a layer whose marker is missing. |
/etc/chalkos/services.json |
The units granted config kinds, from systemd.services.<unit>.chalkos.kinds. |
/etc/chalkos/outputs.json |
The image's outputs, from chalkos.outputs: each one's input kinds, renderer, check, directory, owner and mode. |
/etc/chalkos/validators.json |
The image's validators, from chalkos.validators: each one's input kinds and program. |
/etc/chalkos/effects.json |
What a delivery does to the node's services, from chalkos.effects, with the outputs, the kinds and the rules that fire each effect. |
/run/chalkos/node.json |
The node's identity. Each layer reads its own part below layers; the Kubernetes preparation reads layers.kubernetes. |
/run/chalkos/kubernetes/cluster.json |
The node's Kubernetes cluster, which chalkd builds from the node's Node, Cluster and Flannel documents and image.json at boot and on every apply. It carries the release of the image's kubelet as version and the version images.apiServer names as apiServerVersion; rules read them as facts.cluster.version and facts.cluster.apiServerVersion. |
/run/chalkos/config/<group>/<Kind>/<name>.json |
The node's config documents, Secrets and those of withheld kinds such as Addon excepted, which chalkcfg prints. |
/run/chalkos/credentials/<unit>/<secret>.<key> |
The values the secretRef fields of a unit's kinds name; the unit loads them as the credentials secret_<secret>.<key>. |
/run/chalkos/outputs/<name> |
An output's files, a link to the directory below /run/chalkos/outputs/.rendered that holds the last render; an output with another dir links from there. |
/run/chalkos/outputs/.staging |
The outputs a delivery renders and checks before it swaps them in. |
/run/chalkos/outputs/.status.json |
Each output's hash and last render or check failure, which status reports. |
/run/chalkd/effects-baseline.json |
The config, facts and output hashes the effects compare with while a delivery that failed part way is not applied whole. |
/run/chalkos/storage-status.json |
What the last storage setup found on each disk. |
/run/chalkos/outputs/chrony/servers.sources |
The time servers of the config, for chrony, which the built-in output chrony renders. |
/run/systemd/network |
The networkd units of the config, which the built-in output networkd renders, a link below /run/chalkos/outputs/.rendered. |
/run/chalkos/chrony-dhcp/dhcp.sources |
The time servers DHCP announced, for chrony, when the node's TimeSync takes them with spec.dhcp; empty otherwise. |
/run/containerd/certs.d/<registry>/hosts.toml |
The mirrors of a registry, which containerd reads on every pull, from the node's Mirror and Registry documents. |
/run/chalkos/kubernetes/manifests |
The static pods chalkd renders on a control plane. |
/run/chalkos/kubernetes/pki |
The control plane's certificates and kubeconfigs. |
/run/chalkos/kubernetes/kubelet |
The kubelet's kubeconfig, config.json, flags and ca.crt. |
/run/chalkos/kubernetes/credential-provider |
The kubelet's image credential provider, from the node's Registry documents: its config.json, the credentials in credentials.json, readable by root alone, and the flags that point the kubelet at the configuration and at chalkd kubernetes credential-provider. |
/run/chalkos/kubernetes/node-ip |
The addresses the node picked this boot, the primary family's first. |
/run/chalkos/kubernetes/vxlan |
The addresses and source ranges the firewall's VXLAN rule accepts. |
/var/lib/etcd |
etcd's data on a control plane, whose directory chalkos-controlplane.service makes when the node's Node.spec.type is controlplane. |
/var/lib/kubelet/pki |
The kubelet's client and serving certificates. |
/var/lib/chalkd/failed-boot |
The log lines of the last boot that was not found healthy. |
Sources: modules/node/base/disk.nix for the system region and the device links,
modules/node/base/image-directory.nix for the outputs of an image build,
pkg/config/storage.go for VAR and the volumes, pkg/ociimage for the OCI artifact, and
pkg/esp, pkg/upgrade, pkg/install, pkg/storage and pkg/chalkd for the files on the ESP and STATE
and the paths on a running node.