Bootstrap and get a kubeconfig¶
This page is for an operator whose nodes are installed and waiting for their cluster. One command
starts the cluster on the first control plane, and every other node joins it on its own. At the
end every node is Ready in Kubernetes, and you hold a kubeconfig for the API server. The page
uses the homelab's control planes node1, node2 and node3, behind the VIP
fdc8:a1b2:c3d4:ff::10.
In short:
chalkctl status node1 --secrets=secrets.age
chalkctl bootstrap node1 --secrets=secrets.age
chalkctl kubeconfig --secrets=secrets.age
kubectl --kubeconfig kubeconfig get nodes
Before you begin¶
- Every node is installed and back up in normal mode, as Install a node describes.
- The cluster's
Clusterdocument names the endpoint you chose in Plan a cluster. - You hold the secrets file
secrets.age.chalkctl bootstrapalso works with an admin client file;chalkctl kubeconfigneeds the secrets file. - kubectl is on your workstation, for the check at the end.
A cluster starts once, on one control plane. After that, nothing runs a join command: each node finds the cluster at the endpoint by itself, as chalkctl bootstrap starts the cluster once explains.
Check that the control planes wait¶
An installed control plane cannot tell whether it should start the cluster or join one, so it waits.
chalkctl status shows it in the Kubernetes line:
kubernetes controlplane: waiting for bootstrap or for the cluster at https://[fdc8:a1b2:c3d4:ff::10]:6443
The URL is the cluster endpoint, here the VIP. A worker reads joined once it has
prepared its Kubernetes files, since it has no etcd member to add. Its node ready reads unknown,
followed by the error, until its kubelet has registered its node with the API server.
Bootstrap the first control plane¶
Run chalkctl bootstrap on one control plane:
$ chalkctl bootstrap node1 --secrets=secrets.age
bootstrapping node1; the control plane pulls its images and starts
node1 is bootstrapped; applied 19 objects
chalkd on node1 records the bootstrap on STATE, starts etcd as the cluster's first member and starts
the API server, the controller-manager and the scheduler. It then applies chalkos's add-ons: RBAC,
kube-proxy, flannel and CoreDNS. Last come the objects of the Addon documents. applied 19 objects
counts all of them; the homelab has no Addon documents, so all 19 are chalkos's.
chalkctl waits until the objects are applied, up to --timeout, 20 minutes by default. Most of
that time goes into pulling the control plane's images. When --timeout passes first, chalkctl
stops with a deadline_exceeded error. The bootstrap stays in place and chalkd keeps applying, so
wait and watch chalkctl status node1. Do not bootstrap again: chalkd refuses the second run.
Bootstrap one control plane only. A second bootstrap would start a second etcd cluster, so chalkos refuses it in these cases:
- On a node that is bootstrapped already, chalkd answers
the node is bootstrapped already. - On a node whose cluster answers at the endpoint, chalkd answers
the cluster's API server answers at https://[fdc8:a1b2:c3d4:ff::10]:6443; this node joins it on its own. - On a worker, chalkctl refuses before it sends anything:
node4 is not a control-plane node; bootstrap one of the cluster's control-plane nodes.
Watch the other nodes join¶
Each other control plane finds the cluster at the endpoint, joins etcd and starts its control
plane. etcd accepts one learner at a time, so control planes join one after another. While a node
joins, its status names the step after joining the cluster at <endpoint>:. In the homelab it
reads:
kubernetes controlplane: joining the cluster at https://[fdc8:a1b2:c3d4:ff::10]:6443: checking etcd's members
kubernetes controlplane: joining the cluster at https://[fdc8:a1b2:c3d4:ff::10]:6443: etcd member <id> catches up
<id> is the new member's etcd ID in hexadecimal. A joined control plane reads bootstrapped, the
same as the first one:
kubernetes controlplane: bootstrapped, node ready: True, vip standby, control plane current
kubernetes versions: API server 1.37.2, kubelet 1.37.2, control-plane pods current
vip standby appears when the cluster has a VIP and another control plane holds it; the holder
reads vip holder. Run a highly available control plane covers the joins of
three control planes and what stops one.
A worker's kubelet registers its node at the endpoint with the certificate the worker received at install. Once flannel runs on the node, the node is ready:
Get a kubeconfig¶
chalkctl kubeconfig writes a kubeconfig for the cluster
endpoint, with a client certificate it issues from the secrets file:
$ chalkctl kubeconfig --secrets=secrets.age
wrote kubeconfig; its certificate is valid for 8760h0m0s
The defaults write ./kubeconfig, valid for a year, for the user admin. The file holds the
certificate's private key and is written with mode 0600. chalkctl refuses to replace an existing
file unless --force is given. Other flags change what it writes:
| Flag | Default | Sets |
|---|---|---|
--out |
kubeconfig |
The file to write; - writes to standard output. |
--ttl |
8760h |
How long the certificate is valid. |
--name |
admin |
The user name in the certificate, which the API server's audit log records. |
--context |
the inventory's metadata.name |
The name of the kubeconfig's cluster, user and context. |
--server |
the cluster endpoint | Another URL to reach the API server at, such as a forwarded port. The certificate is still verified for the endpoint. |
--context keeps the names apart when you merge the file into a kubeconfig that holds other
clusters.
Give someone access to the Kubernetes API¶
Issue each person a kubeconfig of their own, with their name and the shortest validity your routine tolerates:
$ chalkctl kubeconfig --name=alice --ttl=2160h --out=alice.kubeconfig --secrets=secrets.age
wrote alice.kubeconfig; its certificate is valid for 2160h0m0s
Every kubeconfig chalkctl issues puts its holder in the group chalkos:cluster-admins, which is
bound to cluster-admin, so each one grants full access to the cluster.
Nothing revokes a single kubeconfig. It works until its certificate expires or the Kubernetes CA is
rotated. Rotate the Kubernetes CAs and keys
takes access away from everyone at once, so a short --ttl lets a kubeconfig lapse without a
rotation.
Narrower access uses Kubernetes RBAC. The API server accepts only client certificates and service-account tokens, as Configure Kubernetes explains. A cluster admin can issue a client certificate for a user and group of their choice through the Kubernetes certificate signing request API, and bind that user or group to a narrower role.
Check that it worked¶
List the nodes with the kubeconfig:
$ kubectl --kubeconfig kubeconfig get nodes
NAME STATUS ROLES AGE VERSION
node1 Ready <none> 4m v1.37.2
node2 Ready <none> 2m v1.37.2
node3 Ready <none> 1m v1.37.2
Every node of the inventory appears, with the Kubernetes release of its image.
ROLES is empty because chalkos sets no node-role label. A node may show NotReady for the first
minute, until flannel runs on it.
chalkctl status shows each node's Kubernetes line, as in the sections above.
If something goes wrong¶
| Symptom | Cause | Fix |
|---|---|---|
A control plane stays at waiting for bootstrap or for the cluster at ... after the bootstrap |
It cannot reach the API server at the endpoint. | Check that the endpoint reaches a bootstrapped control plane on port 6443. |
A worker's node ready stays unknown |
Its kubelet cannot register at the endpoint. | Check chalkctl logs <node> --unit=kubelet.service. |
| The bootstrap times out | The control plane is still pulling its images. | Wait, and watch chalkctl status node1. |
chalkctl kubeconfig fails with no Cluster document of the cluster ... names an endpoint |
The inventory's files give no node a Cluster document. |
Add the Cluster document to the files every node gets. |
Troubleshoot Kubernetes covers further symptoms.
What next¶
- Configure Kubernetes: labels, taints, extra flags and add-ons.
- Run a highly available control plane.
- Add and remove nodes to grow the cluster with more workers.
- Status, logs and the console for the rest of
chalkctl status.