Skip to content

Guard changes with rules and facts

This page is for an image author whose kind has values that must not change freely once a node runs them. At the end the kind carries change rules, chalkctl and chalkd refuse a change that breaks one, and the refusal tells the operator what to do instead. The examples use a Site kind of a web server, myapp, and the Rack kind of rack.nix in the extensions example.

A change rule is a CEL expression that every config the node accepts makes true. The rules are part of the image's schemas, so what a node accepts changes with an upgrade. A rule that needs node state, such as what a survey found, reads it from a fact: the JSON of a file or of a program.

Extension runtime lists the variables and functions a rule reads, its cost limits and its errors. The config pipeline explains where the rules run.

See it in a full extension: Keep a Peer's AS number of the BGP guide keeps a neighbour's AS number and tests the refusal.

Before you begin

You need a kind of your own, as Declare a config kind declares one. Rules on a field need chalkosLib.mkOption, which that page introduces. CEL is the Common Expression Language; its syntax resembles that of C and JavaScript.

Keep a field's value with a field rule

Clients reach a site by its hostname, so a new hostname is a new site. The hostname field of Site declares a rule that keeps it:

hostname = chalkosLib.mkOption {
  type = lib.types.nonEmptyStr;
  description = "Name the site answers to.";
  rules = [
    {
      description = "A Site keeps its hostname.";
      rule = "self == oldSelf";
      messageExpression = ''"the hostname stays " + oldSelf + "; add a Site of another name for " + self'';
    }
  ];
};

In a field's rule, self is the field's value in the delivered document and oldSelf its value in the document of the same name that the node runs. self == oldSelf holds when the hostname stays the same. A rule that reads oldSelf runs only where the node's running config holds the field, so a new Site takes any hostname.

Say the Site shop changes its hostname from shop.example.org to store.example.org. chalkctl config apply fetches the node's config, prints what changed, runs the rule against the node's config, and stops before it sends anything. It prints the node, then the refusal:

the config is refused: Site/shop: spec.hostname: the hostname stays shop.example.org; add a Site of another name for store.example.org

A field rule's refusal starts with the kind and the document's name, then the field, as Site/shop: spec.hostname:, the form a renderer's refusal takes too. The rest comes from messageExpression, which builds the message from the same variables as the rule.

Say what the refused change takes

A rule's message or messageExpression says why the change is refused and what to do instead. Its takes says what the refused change needs instead: refuse, the default, reset or reinstall. Rule attributes lists every attribute of a rule.

A config whose rules fail is refused as a whole. The refusal starts with the most that any failed rule takes, then lists every failed rule's message, separated by ;:

takes The refusal starts with Meaning
refuse the config is refused: The change cannot be made. The message says what to do instead.
reinstall the config takes a reinstall of <node>: Only a reinstall of the node carries the change out.
reset the config takes chalkctl storage reset: chalkctl storage reset carries the change out.

Your own kinds mostly use refuse. The storage kinds use reset and reinstall, because a volume's new format is possible once the volume is recreated. chalkctl storage reset recreates one volume.

Compare a kind's documents with a kind rule

A rule over all documents of a kind goes into chalkos.kinds.<name>.rules. Here documents maps the names of the delivered Site documents to the documents:

chalkos.kinds.Site.rules = [
  {
    rule = "size(documents) <= 2";
    message = "a node serves two sites at most";
  }
];

With three Site documents in a node's files, chalkctl validate refuses the node:

the config is refused: a node serves two sites at most

A kind's message names nothing by itself, so it says which kind it is about.

A value that must not repeat, among a kind's documents or within one list, needs no rule. Declare it with unique on the field or the kind, as Make a field's values unique shows. chalkctl and chalkd check such declarations before any rule runs. A rule that compares each value with every other one costs two nested loops, which comes close to the cost limit.

Compare fields of one document with a spec rule

A rule that compares fields of one document goes into chalkos.kinds.<name>.specRules. Its self is the spec of one delivered document and oldSelf the spec of the running config's document of the same name, so it runs as a field's rule does, once for each document. The Site of Make a field's values unique keeps its backend's address out of its list of clients:

chalkos.kinds.Site.specRules = [
  {
    description = "A Site's clients leave out its backend.";
    rule = ''self.allow.all(c, c.address != document("Backend", self.backend).spec.address)'';
    message = "the backend's address is no client; list the site's clients alone";
  }
];

A spec rule's message starts like a field rule's, with the kind, the document's name and spec: Site/shop: spec: the backend's address is no client; list the site's clients alone.

The same check as a kind rule loops over documents around the loop over the clients. A kind rule that compares two lists of each document, such as each reservation of a subnet with each of its pools, therefore costs three nested loops, which exceeds the cost limit. As a spec rule it costs two.

The build estimates that cost with 256 items in each list, but a node meets the lists of the config it gets. A spec rule over two long lists can pass the build and still exceed the node's limit. Bound such lists with maxItems on chalkosLib.mkOption, so that the schema refuses a list longer than the rule can loop over, and test the rule at the bounds. A DHCPSubnet takes 32 pools and 512 reservations at most: its rule that compares each reservation with each pool then costs about 330,000.

Read node state from a fact

A fact is node state that a rule reads as facts.<name>. The extensions example's Rack kind refuses a location that differs from the one a rack survey wrote to the node:

rack.nix
# The Rack kind, which records a node's rack position, and a unit that reads it on the node. An
# image that carries this module accepts Rack documents. A node that was surveyed, whose
# rack-survey tool wrote where it found the node, takes no other location than that once it runs;
# an install reads no facts and takes any. The comments # --8<-- mark the sections the docs
# include.
{
  chalkosLib,
  config,
  lib,
  pkgs,
  ...
}:
{
  chalkos.kinds.Rack = {
    group = "rack.example.org";
    version = "v1alpha1";
    description = "The rack and height unit a node is mounted in.";
    spec.location = chalkosLib.mkOption {
      type = lib.types.nonEmptyStr;
      example = "rack-a/u12";
      description = "Rack and height unit the node is mounted in.";
      rules = [
        {
          description = "A Rack's location is the one the rack survey found.";
          rule = "facts.rackSurvey == null || self == facts.rackSurvey.location";
          messageExpression = ''"the survey found the node at " + facts.rackSurvey.location + ", not at " + self + "; fix the location, or survey the node again"'';
        }
      ];
    };
  };
  # What the rack survey found, {"location": "rack-a/u12"}; null on a node not surveyed yet.
  chalkos.facts.rackSurvey.file = "/var/lib/rack-survey/location.json";
  # The unit reads the Rack when it starts, so a changed Rack restarts it.
  chalkos.effects.rack-location = chalkosLib.restartOn { kinds = [ "Rack" ]; } [
    "rack-location.service"
  ];
  systemd.services.rack-location = {
    # chalkd writes the node's Rack below /run/chalkos/config for it.
    chalkos.kinds = [ "Rack" ];
    wantedBy = [ "multi-user.target" ];
    serviceConfig = {
      Type = "oneshot";
      RemainAfterExit = true;
      # A path the kind lacks fails the image's evaluation.
      ExecStart = pkgs.writeShellScript "rack-location" (
        config.chalkos.kinds.Rack.get {
          name = "rack";
          path = "spec.location";
        }
      );
    };
  };
}

chalkos.facts declares the fact rackSurvey: the JSON of /var/lib/rack-survey/location.json, or null while the file is missing. The rule does not read oldSelf, so it checks every Rack a node is delivered, its first one too.

Say the survey file on server1 holds {"location": "rack-b/u3"}, because the node moved, while nodes/server1.yaml still says rack-a/u12. The rule reads a fact, so chalkctl cannot check it and the node refuses the delivery. chalkctl prints the node's error after chalkctl: and the error's code. The message part of it is:

the config is refused: Rack/rack: spec.location: the survey found the node at rack-b/u3, not at rack-a/u12; fix the location, or survey the node again

Set the location the survey found where the survey is right, or survey the node again where the document is.

A fact is a file or a command:

  • A file fact, { file = "/var/lib/rack-survey/location.json"; }, is the file's JSON, and null while the file is missing. fields = [ "location" ] keeps only those keys of the file's JSON object, so a rule and its message see nothing else of the file. hashOthers = true keeps the object's other keys too, each as the hex SHA-256 of its value, so a rule sees that a value changed but not what it holds. A file that cannot be read or holds no JSON, or no JSON object with fields or hashOthers, refuses the delivery.
  • A command fact, { command = "${pkg}/bin/tool facts"; }, is a program of the image's store with its arguments, which prints the fact as JSON. Extension runtime lists what it reads, and the sandbox it runs in.

A message can show what a fact holds, as the Rack message shows the survey's location. Keep what a rule does not need out of the fact with fields, or as hashes with hashOthers.

Treat a null fact as unknown, as the Rack rule does with facts.rackSurvey == null. A file fact is null while its file is missing, and facts can be null at an install too: Null means the node does not know lists every case.

Check that it worked

Build the image, which compiles every rule and checks the fields each reads against the kind's schema, as Change rules describes. Then run chalkctl validate, which runs the rules of kinds and the field rules that read neither oldSelf nor facts:

chalkctl validate

The rules that read oldSelf run on chalkctl config apply, against the node's config. Rules that read facts run on the node alone; chalkctl config apply --dry-run has the node run them without delivering the config. A test of the image runs every rule without a node. It gives the rules that read oldSelf its before, and those that read facts its facts, as Test an extension shows.

If it fails

A refusal that starts with cannot check: comes from a rule that cannot be evaluated or a fact that cannot be loaded. It names the kind of error and no value, as Rule errors lists. The journal of chalkd.service holds what a failed command fact wrote.

Built into chalkos

chalkos's own kinds guard changes the same way. Built-ins lists their rules and the facts that they and the built-in effects read: those of every image and those that images with Kubernetes add. The router's DHCPSubnet checks its pools and reservations against each other with spec rules. Built-in rules and facts explains the storage facts and the rules images with Kubernetes add.

What next