Development setup¶
This page sets up a chalkos checkout for development and lists the commands of a development
loop: building the tools and images, running the Go tests, regenerating code and pages,
formatting and linting, and running a lab from the checkout. Everything
runs through Nix, so the only requirements are an x86-64 Linux machine with Nix and flakes
enabled, and /dev/kvm for anything that boots a virtual machine.
First change in 10 minutes¶
These steps take a small Go change from a fresh clone to a commit. The rest of this page explains each command.
-
Clone the repository and enter its development shell. The first run downloads the toolchain:
-
Run the tests of the package you change, here
pkg/inventory: -
Edit the code and its test, then run the same tests again.
-
Format the Go code and the Nix files:
-
Build the flake check of what you touched, with its log. For Go code, that is the linter:
Testing maps each part of the repository to its checks. Leave
nix flake checkfor later: it runs every check, the end-to-end tests that boot VMs included. -
Commit in the form of Conventional Commits, with a type, a scope and a summary in the imperative:
What a change needs lists what a pull request must carry.
Enter the development shell¶
Clone the repository and enter its development shell:
The shell, defined in nix/outputs/shell.nix, provides:
- Go, gopls, staticcheck, buf with protoc-gen-go, protoc-gen-connect-go and protoc-gen-doc, openssl, jq, nixfmt and virt-firmware;
- the tools the tests run: QEMU, swtpm, mtools, sbsigntool, dosfstools, cryptsetup, util-linux, e2fsprogs, vde2, socat and iproute2;
- the variables the tests read:
CHALKLAB_OVMF_CODE,CHALKLAB_OVMF_VARSandCHALKLAB_OVMF_VARS_ENROLLED(OVMF firmware, blank and with test Secure Boot keys enrolled),CHALKLAB_SB_KEYS(those keys),CHALKOS_TEST_EFI,CHALKOS_TEST_REPARTandCHALKOS_TEST_IMAGE_DEFINITIONS.
Tests that need a tool or variable skip without it, so run them in this shell. The flake checks
use the same tools and variables from nix/testing/env.nix.
The Go toolchain¶
The go directive of go.mod names the Go release that builds chalkos, such as go 1.27.4.
The flake input go-overlay provides that release as the Go project publishes it, independent
of the Go version nixpkgs ships. nix/overlay/default.nix selects it from go.mod with go-overlay's
fromGoModStrict, which fails when go-overlay lacks that exact release, and adds it to nixpkgs
as pkgs.chalkos.go, in the scope of chalkos's packages. The development shell, every Go
package, the go-unit and go-lint checks and the programs in the images use it, and the
shell's gopls and staticcheck, which go-lint runs too, are built with it. mkImage applies the
same overlay to the images' nixpkgs, so images built from another nixpkgs still get
chalkos's programs built with this Go.
To move to another Go release:
- Update go-overlay if it predates the release:
nix flake update go-overlay. - Change the
godirective ingo.mod. - Run
nix run .#update-vendor-hash, which setsvendorHashinnix/overlay/build-go-program.nixagain if the Go modules' download changed. - Run the
go-lintandgo-unitchecks and fix their findings.
nix run github:purpleclay/go-overlay#goversions lists the releases go-overlay provides.
Renovate proposes each new Go release in a pull request of its own, and its fixup workflow does
the first and third steps, as Dependency updates describes.
Repository layout¶
| Directory | What lives there |
|---|---|
api/ |
The protobuf definitions of the node API, api/chalkos/node/v1/node.proto, and of each layer's API, api/chalkos/kubernetes/v1/kubernetes.proto and api/chalkos/router/v1/router.proto, and of what chalkd and an image's checker exchange, api/chalkos/check/v1/check.proto, and the template of their reference page. |
cmd/ |
Thin mains: chalkctl, chalkd, chalklab, chalkbuild, chalkcfg, chalkos-console, chalkos-networkd-schema, chalkos-render, chalkos-rules, chalkos-storage and docgen. |
pkg/ |
The Go packages, including pkg/api, the code buf generate writes. |
modules/node/ |
The NixOS modules every image is built from. |
modules/platforms/ |
The metal and kvm platforms. |
modules/installer/ |
The installer image and its ISO. |
modules/testing/ |
The test probe and the test image's small partitions. |
lib/ |
mkImage, mkInstaller, mkImages and the module of their arguments, and the layers, the flake's nixosModules. |
nix/ |
Tooling: packages, apps, checks, the development shell, the docs build, and the test images and their secrets in nix/testing/. |
templates/lab/ |
The lab template the quick start uses. |
templates/router-lab/ |
The router lab template the router quick start uses. |
templates/cluster-metal/ |
The template of a cluster on real machines, which the guide Build a cluster on real machines uses. |
examples/homelab/ |
The router and Kubernetes node images with built-in BGP, the installer, and the inventory and YAML for Build a home lab. The example-homelab check builds the images and validates the inventory. |
examples/bgp-extension/ |
The BIRD extension, its example config and golden files, and the node-bird-metal image for Write a BGP extension. |
examples/extensions/ |
An example of extension kinds: a flake that builds an image with the Rack and Hardware extensions and Kubernetes off, an inventory and its node's YAML, which the example-extensions check builds and validates. |
test/e2e/ |
The end-to-end tests, which boot virtual machines. |
test/nix/ |
The Nix evaluation tests. |
test/veritysize/ |
A helper of the image-size check that reads an image's store and hash tree sizes with pkg/verity. |
test/fixtures/ |
Static test data: the e2e clusters' YAML, kind schemas and config documents. |
docs/ |
The documentation site's pages. |
Packages and programs describes the Go packages and programs one by one, and Nix code the modules.
Build the tools¶
The flake's packages build the programs with Nix, as a user gets them:
chalklab is wrapped with QEMU, OVMF, swtpm and the other tools a lab runs, and with
chalkctl, so its closure is larger than chalkctl's. chalkbuild is wrapped with mtools and
sbsigntool, which it signs images with. During development, run the commands from the working
tree instead:
chalkd only does useful work on a node; the end-to-end tests and a lab run it there.
Build images¶
A cluster's flake builds its images with chalkos.lib.mkImages, which returns each image by its
name, and exposes them as packages. The repository's own flake exposes its test images, which
nix/testing/images.nix builds, as packages:
| Package | What it is |
|---|---|
test-image |
The image test-metal, with small partitions and the test probe; most VM tests boot it. |
test-installer |
The test images' installer. |
test-kubernetes-image |
The image k8s-kvm, with the Kubernetes layer, which the Kubernetes test's control plane and worker boot. |
test-kubernetes-ha-image |
The image k8s-metal, without BGP, which the HA test's control planes boot. |
test-router-image |
The image router-kvm, with the router layer, which the router test's routers boot. |
test-upgrade-image, test-unhealthy-image, test-kubernetes-ha-upgrade-image |
Images the upgrade tests install. |
installer |
An installer of a cluster without an OS CA, which accepts any client until it installs a node. |
The result is a directory with the raw image, repart-output.json, repart.d, metadata.json
and schemas, as
Image and partition layout lists.
Run the Go tests¶
Run the unit tests in the development shell:
None of them needs root. The tests of pkg/kubernetes/vip and pkg/kubernetes/nodeip run
themselves again in a new user and network namespace, and a test of pkg/kubernetes/node in a new
user and mount namespace, so the kernel must allow unprivileged user namespaces. The install and
upgrade tests lay out disk image files with the real sfdisk and systemd-repart and emulate
mounts, LUKS, the TPM and the firmware, so they need no loop
devices.
go test ./... also runs the end-to-end tests in test/e2e, which boot VMs as soon as the
variables they need are set; the development shell sets those of the firmware test. Run them
through their flake checks instead, as Testing describes.
Regenerate code and pages¶
The Go code of the node API and of each layer's API, and their reference page, come from the
proto files below api/chalkos/:
The reference pages for the commands, the options and the API come from the code:
Both write into the working tree, and the results are committed. The api-generated and
docs-generated checks fail when they are out of date.
Format and lint¶
nix fmt runs nixfmt over the Nix files; nix fmt -- --ci checks them without changing
anything.
Lint the Nix code with statix and with deadnix, which finds unused bindings and arguments:
statix.toml lists the statix lints chalkos leaves out, each with its reason. The nix-lint check
runs nixfmt, deadnix and statix in the Nix sandbox and fails on any finding.
Lint the Go code, the end-to-end tests included, with go vet and staticcheck:
staticcheck reads staticcheck.conf at the root of the repository, which enables all its checks
but a few and gives the reason for each one it leaves out. A finding that is wrong for one line
takes a //lint:ignore <check> <reason> comment above that line. The go-lint check runs gofmt,
go vet and staticcheck in the Nix sandbox and fails on any finding.
Run a lab from the checkout¶
A lab runs a cluster's nodes as QEMU VMs with Secure Boot and a TPM, which is the quickest way
to see a change on a booted cluster. Create one from the template, pointing its chalkos input
at the checkout:
<checkout> is the absolute path of the repository. In the lab's flake.nix, replace the
input's URL:
Then follow the template's README, with chalklab and chalkctl now built from the checkout:
git init && git add .
nix develop
chalkctl gen secrets --plaintext
git add . && git commit -m "A chalkos lab"
chalklab create
A git+file input reads the files git tracks, so add new files in the checkout before the lab
sees them. After changing the checkout, run nix flake update chalkos in the lab to lock the new
state, then upgrade the lab's nodes as the template's README shows, or recreate the lab with
chalklab destroy and chalklab create. The quick start walks
through a lab step by step.
Serve the documentation¶
This serves the site from the working tree and rebuilds it as pages change. Writing documentation covers the rest of the documentation's tooling.