More useful facts generation

Materia now gathers the facts it needs to generate plans in a much more standardized manner: Facts are gathered by collectors and organized into namespaces. The number of built in facts have also been expanded to cover information like “what components are installed” and “what was the last source revision applied and was it successful”?

Existing facts, such as the current hostname and network interface(s) available have been moved into the host namespace.

The following facts are now available:

host.hostname: The hosts reported (not configured in Materia) hostname

host.interfaces: All network interfaces on the host and their IPv4/IPv6 addresses

materia.hostname: The configured hostname, if any

materia.roles: What roles the host has

materia.assigned_components: What components are assigned to the host

materia.installed_components: What components are already installed

materia.last_revision: The last source revision used in an update operation

materia.last_run: When the last update opperation occured

materia.last_run_successful: Whether the last update operation was successful.

As always, these facts are accessible via the materia facts commands or the facts Varlink call for the server. The next step for fact generation is to allow custom external fact collectors.

As an internal note, anything gathered as a fact is now gathered only at the start of a Materia plan. For most users this shouldn’t affect anything, but it does make plan generation much more deterministic, so if you had issues with, say, network interfaces coming and going during plan generation that should happen no more!

Consistent Output Data

Those new source revision facts didn’t come from nowhere: Materia now saves output data on each plan and run to $MATERIA_DIR/output for your future perusal and consumption by external tools.

Each materia plan and materia update operation, whether via command line or server mode, will now generate a lastplan.toml and lastrun.toml file, respectively. Up to five instances (customizable in the future) of each will be saved; older files will be rotated to filename-x.toml. For example, if lastrun.toml already exists in the output directory, the next run will be saved as lastrun.toml and the existing run will become lastrun-1.toml.

These files are parsed by materia to generate the last_revision/last_run//etc facts, so be aware that if you delete them you won’t have those facts until the next run.

Server mode improvements

Server mode continues to get many improvements to bring it in-line with the existing CLI materia plan-materia upate loop.

• Server mode now uses the exact same internal calls as the CLI; no more strange mismatches between the two!
• Server mode webhook management has been expanded:
  • You can now pass the security token as a Bearer: token instead of in the /update payload
  • A new /facts endpoint that you can GET. This is useful for validating current deployment status
  • There’s now a /health endpoint that gets automatically activated when any other endpoint is activated
• Rollback now works the same for server mode as it does for the CLI. Note: this is still dependent on individual source support.

The Varlink API has also been updated to match the new facts command line arguments.

File management improvement

Materia can now manage the permissions of files it manages, such as its Quadlet and data files. This is included along with many other changes¹ that should help prevent any path escapes from happening during Materia execution.

If you haven’t been running latest, this will be visible as a big wave of Update actions being taken. This will only happen once as Materia fixes all the file permissions that lazily got installed incorrectly.

A new release pipeline

Materia releases have been re-organized into binary tarballs instead of raw binaries and are now signed via Cosign upon CI release. Release images are also signed via Cosign; please note that as :stable just points to the latest tagged release, it does not have its own signed image digest.

The public key for verifying the signed releases can be found in the repository root and on this website’s root.

The new binary tarballs also include the README and generated man pages for your convenience.

Changelog

- feat: most materia packages are now public. This will be stabilized and documented better in 0.9.
    - This may also fix some bugs in server mode that (hopefully) no-one has noticed
- feat: git repos can reset on corrupted HEAD
- feat: git source supports `git.auth_token` for `Bearer: <token>` HTTP auth
- refactor: rewrote file management backend
    - Files are now installed with consistent permissions and using `os.Root` to prevent symlink/path escaping
    - You may notice an initial wave of `Update` actions as materia fixes installed file permissions. Note that changing permissions does count as a normal update action, so all triggered effects (container restarts,etc) will still occur
- feat/refactor: The `materia` container image is now built from `podman/stable` instead of `tumbleweed/bci:latest`
- ci: release tarballs and images are now signed via cosign. Public key for validation is in the repo and the main primamateria.systems website
- feat/rework: Redid `facts` generation. Facts are now denoted in dotted-path format.
    - Existing facts and their new versions:
        - `hostname` -> `materia.hostname`
        - `interface` -> `host.interfaces`
    - New facts
        - `host.hostname`: hostname as the OS sees its
        - `materia.roles`: Assigned roles
        - `materia.assigned_components`: Assigned components
        - `materia.installed_components`: Installed components
- feat: generate a $OUTPUTDIR/lastrun.toml file on `materia update` or `materia server update`. Up to five last-plan and last-run files are stored.
    - If you have existing `lastrun.toml` files, you need to remove them before running `materia facts`.
- feat: facts `materia.output` now shows last applied revision and timestamp and whether it was succesful:
    - New facts:
        - `materia.last_revision`
        - `materia.last_run`
        - `materia.last_run_succesful`
- feat: server mode supports new `server.token` for `Bearer: token` style authentication:
    - this replaces the now deprecated `server.update_secret`.
- feat: other server config changes:
    - `server.url`: replaces `server.update_url`
    - `server.update_webhook` requires `server.update_secret` or `server.token`
    - `update_secret` or `token` checks are disabled by setting `server.url` to `localhost*`
- feat: new `/facts` endpoint for server mode. Enable with `server.facts_endpoint`  Requires `server.token` to be enabled.
- feat: new `/health` endpoint, automatically enabled when any other webhook is enabled or if `server.health` is `true`.
- bugfix: `lastplan`/`lastrun` files can no longer include un-redacted podman secrets. Thanks shirome on Matrix!
    - The planner no longer contains Podman secrets contents at all, even in memory, so this should not happen again
- feat: server and cli use same internal commands. Server now supports rollback
- chore: upgrade to Go 1.27.3
- feat/chore: rework `materia validate` to use new facts system

Deprecations

1. The file source has been renamed to local to prevent settings collisions with the file attributes engine. Setting the SOURCE_KIND to file will work until after 0.8 is released. This is a straight rename; no other settings to be adjusted besides replacing file with local.
2. The following server settings are deprecated:
   • server.update_url: replaced with server.url
   • server.update_secret: replaced with Bearer token auth with server.token
3. The following facts are deprecated:
   • hostname: replaced with host.hostname
   • interface: replaced with host.interfaces

Rollback on failed updates

0.7.0 adds the first steps for fullly self healing quadlets. By setting the new rollback.kind setting to service in your config, materia will now detect when an update results in one or more services in the failed state and will “rollback” its local source cache to the state before the update, running a new plan-execute cycle along the way. This will also send a “Rollback initated” message to a notification webhook, if configured.

This service healthcheck occurs at two points during an update: when the service’s state is changed (i.e. when materia runs systemctl start foo.service) and at the end of the execute portion, where Materia will verify all adjusted services are in the state they should be.

Currently this only works with the git source type, but I plan for oci support to be added in a point release or two once any initial bugs are ironed out.

As some of you may have guessed from this getting its own [rollback] configuration block, I plan on expandeding this feature in the future with multiple types of rollback healthchecks, support for using this in server mode, and different rollback types, such as migrating volume state.

This has been one of the most asked about features on the roadmap.md for a while now. In fact, I think this is the only feature asked for by multiple independent people; this shows both how much people want it and the variety of ways people use Materia :)

Server mode improvements

The materia server command has been given a little refresh this update.

The biggest change is server mode now starts a Varlink-compatible server on its listening socket. This replaces the existing bespoke command setup, and was done for three reasons:

1. systemd is moving to Varlink as its IPC system and Materia strives to be compatible with it
2. Varlink makes it easy to expose the same API both locally and over HTTP with varlink-http-bridge
3. Having a standardized protocol should hopefully make it easier for you to automate Materia in more expansive ways.

You can use the materia agent command to try it out, or introspect the socket directly with varlinkctl.

Additionally, server mode now supports starting a /webhook endpoint for triggering materia update on command with a JSON payload. Server mode also now uses the new notify substem for sending alerts, keeping it in line with the rest of Materia

Instanced Components

Systemd supports templating service units by naming them with the format servicename@ and running them with systemctl start servicename@foo. This is useful for running multiple instances of the same server with different configurations. For example web-server@site1 and web-server@site2 use the same web-server@ unit file.

You can now use Materia components in the same way; simply assign the component to the host/role with the same component@instance format as systemd expects. Materia will then detect any name@ style resources and install them with the proper instance variable like name@instance instead, allowing them to be used as systemd instanced services.

This also applies to attribute expansion: Materia will search for both component and component@instance attributes at run time, with the latter overriding the former.

Other Changes You Should Know

Changelog

- feat: Components with instanced systemd units (i.e. `unit@.service`) can now be instanced at the component level
- refactor: removed `autoUpdate` snippet, added `onBoot` and `harden` snippets.
- feat: Added `m_quadletDir`, `m_outputDir`, `m_scriptsDir`, `m_serviceDir` macros
- feat: simplified volume compression. `containers.compression_command` and `containers.compression_suffix` have been simplified into one `containers.compression` setting that can be blank, `gzip`, or `zstd`
- refactor/note: volume dump file are now in the format `volumename-volume.tar(.gz/zst)`.
- feat: `PreCommand` and `PostCommand` component manifest settings for running scripts before or after an update. Experimental.
- feat: Remote components can specify a `subpath` to use.
- tests: new testcontainers based setup to replace virter tests
- feat: `clean` command now attempts to gracefully remove components. Skippable with `--force`.
- feat: server mode can now listen on a webhook for sync/update requests.
    - `server.webhook` is now `server.notify_webhook`, though see also the new `notify` config block
    - `server.update_webhook` = true|false
    - `server.update_secret` = "shared secret"
    - `server.update_url` = URL to listen on. Default `:6284/webhook`
- feat: `lock` setting causes materia to create a dbus or file lock, preventing multiple instances from running destructive actions.
- feat: `materia server` now exposes a varlink api over its socket
- feat: `materia agent` now uses new varlink api
- refactor: materia now uses native podman bindings instead of wrapping the `podman` command. If you experience issues with this, use the `MATERIA_PODMAN_COMMAND` flag to restore the old behaviour.
- feat: quadlet drop-ins are now left-unchanged by `materia update`.
- feat: new service resource config setting: `Oneshot`.
- feat: experimental rollback on failure support. Enable with `rollback.kind = "service"`
    - service state health checks have been rewritten
    - Currently supports git source, OCI coming post 0.7.0
    - If a materia update leads to a failed service or a service is in the wrong state at the end of the update, checkout the last known git commit and run the update again
- feat: `notify` config block for configuring all webhook notifications in one place
    - Set `notify.triggers.update` to replace `server.notify_webhook`
    - Get notifications when rollback occurs with `notify.triggers.rollback`
    - `server.notify_webhook` is now deprecated and will be removed in a future release, server mode uses same notification code as everywhere else.
- refactor: removed `source.URL` guessing. `source.URL` is now passed directly to the sources config.

Deprecations

1. { snippet "autoUpdate" } has been removed; there was little purpose for it with AutoUpdate= already existing.

2. Materia will no longer attempt to guess the source type from source.URL. You now need to specify the source type with source.kind.

The Premise

The quick bullet list:

• I have a static website in a private repo on Github
• I need git-lfs to clone it
• I don’t want to install anything on my MicroOS server since that would require a transactional upgrade
• BONUS I already have caddy running on the server and it has /srv/www ¹ already bind mounted.

That last part about Caddy is optional; I could very easily deploy it as part of this pipeline as well but I happened to have it already running for my other static sites.

Building on the host

To download the website onto the host I need two things: a script that runs the git commands and an image with git and git-lfs in it.

Fortunately, there’s a Quadlet type that fits my exact need: .build units are Quadlets that build a Podman image on the host. You can think of them as declarative podman build commands in a way.

We need an environment to grab the site in. We need two tools: git and git-lfs, plus an SSH key to clone it with.

First off, here’s the script that I’m going to use to clone the website:

Sync script sync_sh²:

#!/bin/sh
set -eu

: "${REPO_URL:?REPO_URL environment variable is required}"
: "${SYNC_DEST:?SYNC_DEST environment variable is requires}"

if [ -d "$SYNC_DEST/.git" ]; then
    echo "Updating"
    git -C "$SYNC_DEST" fetch --all
    git -C "$SYNC_DEST" reset --hard "@{u}"
    git -C "$SYNC_DEST" lfs pull
else
    echo "Cloning"
    git clone "$REPO_URL" "$SYNC_DEST"
    git -C "$SYNC_DEST" lfs pull
fi

echo "Sync complete: $(date -u)"

Since this is running fully automated I’m using git fetch and git reset to ignore any local changes that might get made by accident. This script gets added to the final image below.

Next we have the Containerfile:

FROM alpine:latest

RUN apk add --no-cache \
    git \
    git-lfs \
    openssh-client

RUN git lfs install

RUN mkdir -p /srv/www
RUN mkdir -p ~/.ssh && ssh-keyscan github.com >> ~/.ssh/known_hosts

COPY sync_sh /usr/local/bin/sync.sh
RUN chmod +x /usr/local/bin/sync.sh


ENTRYPOINT ["/usr/local/bin/sync.sh"]

A very simple image! We could even do everything in the container and not embed a separate script, but this is easier to maintain. The one adjustment I’d like to make later is to not set up known_hosts in the image since that could get out of date quickly, but this seemed like an easy duck-tape for now.

Finally, we need a .build Quadlet to automate the image building process.

site-builder.build.gotmpl

[Build]
File=Containerfile
ImageTag=localhost/site-builder:latest
SetWorkingDirectory={{ m_dataDir "staticsite" }} # evaluates to /var/lib/materia/components/staticsite

Since this is being deployed with Materia I’m using the m_dataDir macro to keep things dynamic, but there’s nothing stopping you from hard-coding the value.

That takes care of actually downloading the static site and building the image. Now I need to make it run on a regular basis.

Automating the build

I need two things for this: a container quadlet and a timer

The site-updater.container.gotmpl quadlet:

[Unit]
Description=static site updater

[Service]
Type=oneshot

[Container]
Image=site-builder.build
Volume=/srv/www:/srv/www:z

Secret=materia-github-ssh,type=mount,target=/root/.ssh/id_rsa,mode=0600

Environment=REPO_URL={{ .repoURL }}
Environment=SYNC_DEST={{ .dest }}

The key here is having the Image= setting be the build unit directly instead of the resulting image. This tells the Quadlet systemd-generator that the resulting service file for the container needs the image built first, automatically adding Wants=site-builder and Requires=site-builder as service dependencies.

I’m mounting a Podman secret into the container with the Secret= option³. This will mount whatever data is stored in the materia-github-ssh secret as a file in the container. I’m also once again we’re using Materia’s templating support to dynamically insert the REPO_URL and SYNC_DEST values, but there’s nothing stopping you from hard coding them.

I manually specified it as a one-shot service so we don’t intend for it to be running all the time, only once to run the embedded script.

Finally, I need a timer to automatically run the container once a day.

site-updater.timer:

[Unit]
Description=Static Site daily update

[Timer]
RandomizedDelaySec=1800
OnCalendar=*-*-* 16:00:00

[Install]
WantedBy=timers.target

This is pretty standard timer. Since .container Quadlets get converted into .service units I don’t need to specify it with Unit= here.

Tying it all together

Since I’m deploying this as a Materia component we naturally have a MANIFEST.toml file to describe how its deployed:

Secrets = ["github-ssh"]


[[Services]]
Service = "site-updater.timer"
Static = true

This manifest does two things:

1. It declares that the github-ssh attribute should be stored as a Podman secret on the host (and thus made available in the Container with the above Secret= line)
2. The site-updater.timer should be enabled and started.

Both of these steps can easily be done manually if desired, but keeping it as a Materia component lets me move it to another host easily by just changing my repositories manifest file.

The Final Result

The final Materia component looks like this:

materia_repo/components/staticsite master ❯ ls -la
total 32K
drwxr-xr-x.  2 stryan stryan 4.0K Jun  2 21:00 .
drwxr-xr-x. 49 stryan stryan 4.0K Jun  2 20:38 ..
-rw-r--r--.  1 stryan stryan  119 Jun  2 20:40 site-builder.build.gotmpl
-rw-r--r--.  1 stryan stryan  320 Jun  2 21:17 site-updater.container.gotmpl
-rw-r--r--.  1 stryan stryan  136 Jun  2 20:43 site-updater.timer
-rw-r--r--.  1 stryan stryan  368 Jun  2 20:45 Containerfile
-rw-r--r--.  1 stryan stryan   97 Jun  2 20:49 MANIFEST.toml
-rw-r--r--.  1 stryan stryan  465 Jun  2 20:43 sync_sh

On the server itself, running the actual site-updater.service does this:

1. Systemd sees site-updater.service depends on site-builder.build
2. site-builder.build is started
3. The localhost/staticsite:latest image is build with the embedded shell script. This will be quick since all the layers are cached.
4. site-updater.service runs, starting a localhost/staticsite:latest container with the github-ssh secret mounted inside as an SSH key and /srv/www bind-mounted.
5. The script runs, downloading the static site git repo to /srv/www/<site-destination>.

Was this worth it?

Maybe! If you’re not running an atomic distro like MicroOS or Fedora CoreOS, it’s probably a little more effort then its worth. The way I see it though, this workflow has two big benefits:

1. It keeps all the work done on the host itself; I used to deploy my sites with a CI pipeline, but that meant Yet Another Server to manage deploy keys and workflows on.
2. Writing it as Quadlets keeps the whole thing essentially OS agnostic. I already know all my machines have Podman and Systemd on them, so I can move this setup to any other server I want without having to worry about having the git-lfs dependency.

So I’m pretty happy that I went with this instead of a bespoke cron + shell script setup.

There’s a few other improvements I could make to this workflow as well, like container health checks to make sure the site actually syncs properly and moving to image backed Volumes instead of a bind mount. The site-builder.build, site-updater.container pattern is also an obvious candidate for being moved into an instanced component so I can have multiple copies of it like static-site@quadlets-are-cool, static-site@nerf-seven, etc.

Finally, you may have noticed that, if you squint at this hard enough, this is basically a small build/CICD system powered by systemd. I’m not going to say this is a good usage of systemd job scheduling abilities, but if you ever needed a job system that’s already on your host, you could do worse then systemd ⁴.

OCI images as a repository source

You can now specify a remote OCI image to use as your Materia repository! Simply create a bare image with the normal Materia repository file layout at / and pass it as the source URL like any other image: MATERIA_SOURCE__URL=oci://git.saintnet.tech/stryan/materia-example-repo:latest. The example repository has been updated with a mise task that demonstrates how to do so. This is useful if you’re building your repository with some automated process like Nix, or you simply don’t want to expose your Git forge over the internet.

This feature was contributed by TECHNOFAB on Github.

Quadlets and AppMode compatibility

Podman 5.8 introduced a new Quadlet type: .quadlets. These files contain multiple quadlets concatenated together, an example of which can be seen on the official Podman documentation. Materia now officially supports this quadlet type with its own implementation, meaning you don’t need Podman 5.8 to use it. Quadlets files can be included in your components and repositories like any other Quadlet type and Materia will automatically expand and install the file as its “component” Quadlet files.

On a related note, Materia now has a MATERIA_APPMODE (default false) configuration option that causes it to generate .component_name.app files for installed components. This is too improve compatibility with the podman quadlet commands, as now you can see which quadlets are a part of what component. Please note that since Materia stores data files in its own folder the app file does not contain the data files associated with the component, only the Quadlet files.

Component Script improvements

Did you know? Components can be scripted with post-install and post-removal steps, for all those tricky installation steps that need to be done imperatively or are too tedious to implement as systemd units.

Originally this feature was implemented using special setup.sh and cleanup.sh scripts included with the component, but now is done entirely through the Component manifest file and systemd one-shot jobs. Normal script resources can be specified in the new Settings.SetupScript and Settings.CleanupScript Component manifest options. When the component is installed Materia will run the specified setup script as a systemd transitive job with the name componentname-materia-setup.service. Conversely, when the component is removed Materia will run the specified cleanup script as componentname-materia-cleanup.service. These jobs are set to remain after exit so you can examine logs if needed, but running the startup job will attempt to remove any stale cleanup jobs and vice-versa.

Changelog

- refactor: planner,plan,containers,services, executor, and component loading modules have been publicized and refactored.
- feat: Component scripts are no longer a separate resource type. They are instead normal script resources that, when specified in the component manifest, are run in a transient systemd unit on installation/removal.
- bugfix: component removal ignores non-existent services
- bugfix/feat: `Overrides` now properly overrides manifests, new `Extensions` manifest option
- feat: add support for OCI images as repository sources.
- feat: experimental support for `.quadlets` files
- feat: LoadAllAttributes config setting for attributes engine to ignore host/role filtering
- feat: new `services` config block
- feat/refactor: remote components use new source format
- feat: `appMode` config option to generate `podman quadlet` compatible app files

Deprecations

As mentioned in the previous release, the cleanup, cleanup_volumes, backup_volumes, migrate_volumes settings are now in the planner config section.

Planner re-work and new settings

Much of this release was focusing on improving the planner which, as you may have guessed, is the big block of code that takes the current state of the host and repository as inputs and figures out the plan it needs to take to get the host reconciled with the desired state. Naturally, the planner is probably the oldest and most complicated chunk of code in the codebase. Unfortunately it was so complex it was making it complicated to add fun new features like dynamic timeouts so I refactored and separated out a bunch of the functions into less complex units. The user-facing changes from this should be minimal.

On my continued journey to clean up Materia’s configuration, I also separated out config options that directly affect plan generation into their own table. The following settings were migrated to the new planner section, with the old locations to be deprecated in 0.6:

• cleanup -> planner.cleanup_quadlets
• cleanup_volumes -> planner.cleanup_volumes
• migrate_volumes -> planner.migrate_volumes
• backup_volumes -> planner.backup_volumes

On a related note, the old cleanup function used to also purge components after they failed to install. I don’t think anyone ever used this feature but if you did it’s now under executor.cleanup_components.

Service definitions can use Quadlets

It felt silly to make users have to figure out what service name a Quadlet will translate to, so now Materia can use direct Quadlet resource names in the [[Services]] array.

Dynamic timeouts

Materia now automatically adjusts the timeout it uses for systemctl actions to account for whether the service is dependant on another object. This was specifically added to account for situations like .container Quadlets using Image=otherimage.build: since building a Container can take a while, the default 30s timeout was far too little but adjusting the overall timeout felt off since you’d still want other services to fail in a normal amount of time. Now Materia will A) automatically add some extra time to the timeout free of charge when it detects this situation, B) allow you to manually specify timeouts for individual Services and C) combine Service timeouts when required.

For example: if hello.container relies on hello.build, you can define the following Services for them in the manifest: ~ini [[Services]] Service = “hello.container”

[[Services]] Service = “hello.build” Stopped = true Timeout = 100 ~

Materia will add hello.build’s 100s timeout to any service actions applied to hello.container.

As part of this, you can also specify Stopped in your [[Services]] entries to tell Materia not to start a service automatically, as used in the .build unit above.

Ensuring Volumes and Networks

While testing dynamic timeouts I noticed something annoying about how .volume Quadlets worked: Since Podman creates those as one-shot units, running them again won’t actually do anything if they’re been run once. This is especially annoying when you do something like remove a Volume quadlet in one update, delete the Podman volume, and re-add it in another since systemd will just see the still active volumename-volume.service and not do anything.

This annoyed me so much that Materia now checks for this scenario and plans an Ensure action to account for it; if it detects a .volume/.network/.image resource being updated and doesn’t see an equivalent Podman resource for it, it will ensure it exists.

Changelog

- refactor: planner has been heavily refactored
- feat: Quadlet resources (.container,.pod,etc) can be used as services in Component services configurations
- feat: Containers that rely on Build or Image quadlets will use dynamic timeouts:
- feat: New `planner` and `executor` config sections, deprecating the old `cleanup`,`cleanup_volumes`,`migrate_volumes`,`backup_volumes` config option locations
- feat: ensure volumes and networks exist when unit already started; this fixes situations like installing,removing, and reinstalling a volume.
- feat/bugfix: materia will no longer attempt to cleanup quadlets that are still in use by other containers

Creating the directory layout and manifest

First off, let’s pop into my materia repository to create the main component directory and manifest. I like to create the manifest first since a component is never valid without one.

We’ll run all these commands from the root of my repository.

> cd code/materia_repo
> ls
components  secrets  MANIFEST.toml  mise.local.toml  mise.toml

(The mise files are mostly just tasks for easy sops encryption)

Let’s create the folder and manifest file for the dnsmasq component:

> mkdir components/dnsmasq
> touch components/dnsmasq/MANIFEST.toml
> cd components/dnsmasq
> ls
MANIFEST.toml

I like to fill in the manifest last when writing components, so we’ll write up the container file first.

Writing the .container file

We’ll create a dnsmasq.container file first:

components/dnsmasq/dnsmasq.container:

[Container]
Image=docker.io/dockurr/dnsmasq:latest
ContainerName=dnsmasq
PublishPort=53:53/udp
ReloadSignal=SIGHUP
Volume=/etc/dnsmasq:/etc/dnsmasq
AddCapability=NET_ADMIN
AutoUpdate=registry

[Install]
# Needed to make container start at boot
WantedBy=multi-user.target default.target

This is essentially the simplest quadlet we could write: it will create the dnsmasq container and mount the host’s /etc/dnasmasq directory in it. If materia was run with this component right now, the only thing it would do is install this container file as-is to /etc/containers/systemd.

Realistically though we’ll want more than this. We want to supply configuration files as well as the container Quadlet and allow customization in case we need to run it on hosts with differing needs.

We probably also want to have a nice description and restart automatically if it fails.

Let’s convert it into a template and make it more flexible. Replace the dnsmasq.container file with this dnsmasq.container.gotmpl file:

components/dnsmasq/dnsmasq.container.gotmpl: ~ini [Unit] Description=dnsmasq container StartLimitIntervalSec=400 StartLimitBurst=3

[Service] Restart=on-failure RestartSec=5s

[Container] Image=docker.io/dockurr/dnsmasq:{{ .containerTag }} ContainerName=dnsmasq PublishPort={{ .hostPort }}:53/udp ReloadSignal=SIGHUP Volume={{ m_dataDir “dnsmasq” }}/dnsmasq.conf:/etc/dnsmasq.conf:Z {{- if .useSeperateFiles }} Volume={{ m_dataDir “dnsmasq” }}/hosts:/etc/hosts:Z Volume={{ m_dataDir “dnsmasq” }}/ethers:/etc/ethers:Z {{- end }} AddCapability=NET_ADMIN AutoUpdate=registry

[Install] WantedBy=multi-user.target default.target ~

See the differences? Here’s a brief summary of what changed:

1. We added [Unit] and [Service] sections. Like the [Install] section these are normal systemd unit properties that will apply to the generated unit.

2. We added two simple variables: { .containerTag } and { .hostPort }. These will be templated in with values from your configured attributes vaults, or from the defaults section of the component manifest. Spoilers: We’re going to set them in the component manifest.

3. We added an if-statement with {- if .useSeperateFiles }; since these are normal Go Templates we can use any feature available there. I’m old fashioned and like having dnsmasq use /etc/hosts and the oft-forgotten /etc/ethers files, but some situations or users might want just the one dnsmasq.conf file.

4. We used the m_dataDir macro; these are built-in functions for materia that provide useful features. In this case the m_dataDir "component_name" macro resolves to data directory for the component_name, letting us refer to data files in a component without hardcoding references. In this case, it would be /var/lib/materia/components/dnsmasq/.

5. Finally, we used :Z to get around selinux permissions issues; feel free to skip this step if you don’t use selinux. If you don’t include it you may see errors in the logs about not being able to access the data files.

Now we have a single container file that’s more robust and flexible thanks to systemd and templating. Since there’s no other quadlet’s needed here (i.e. we’re not creating any podman volumes or networks) we’ll move to the data files.

Adding data files

We need to create three data files: dnsmasq.conf, hosts, and ethers. If this component was being designed as a remote component we would make this part with more robust templating, but since that would be an article it’s own right we’re going to be a little lazy and create the following three files as such:

components/dnsmasq/dnsmasq.conf.gotmpl: ~gotmpl {{ .confContents }} ~

components/dnsmasq/hosts.gotmpl: ~gotmpl {{ .hosts }} ~

components/dnsmasq/ethers.gotmpl: ~gotmpl {{ .ethers }} ~

These files will be filled with the contents of their respective attributes.

This setup gives us maximum flexibility with minimum blog post length :)

Writing the component manifest

This brings us to the final file, the component manifest. We need to do two things here: set the default values for all the variables we used and configure what services should be running.

components/dnsmasq/MANIFEST.toml: ~toml [Defaults] containerTag = “latest” hostPort = “53” useSeperateFiles = true confContents = “” hosts = “” ethers = “”

[[Services]] Service = “dnsmasq.container” ReloadedBy = [“hosts”,“ethers”,“dnsmasq.conf”] ~

We’ve created two sections here: [Defaults] and [[Services]].

1. [Defaults] is a table that defines the default attribute values for this component. For example, the containerTag attribute will be templated as “latest” by default, unless overridden somewhere else in your repository (usually as a host attribute). You usually always want to set a default value for attributes, unless you specifically do not want the component to be useable without the user setting one. We’re going to set everything for dnsmasq this time, but an argument can be made for not setting confContents to force the user to provide a configuration.

2. [[Services]] is an array of service definitions. Materia (unless configured otherwise) starts any systemd units or quadlets defined here as part of the component installation and checks to make sure they’re running whenever it updates. Here we defined the main dnsmasq service by telling it to start the dnsmasq.container quadlet (which it will translate to the proper systemd unit). We also defined the dnsmasq.container service to be reloaded by the hosts, ethers, and dnsmasq.conf resources. That means whenever any of those files are updated, materia will trigger a systemctl reload for the dnsmasq service.

The resulting file layout for the component should look like this: ~bash > ls components/dnsmasq dnsmasq.conf.gotmpl dnsmasq.container.gotmpl ethers.gotmpl hosts.gotmpl MANIFEST.toml ~

That’s it! The dnsmasq component is now complete and ready for use.

Adding it to a host

Just to demonstrate, we’re going to assign this component to my server named viscous. We’ll do this by editing the main MANIFEST.toml file at the root of the repository and adding it to the host config for viscous.

MANIFEST.toml: ~toml [hosts.viscous] components = [“dnsmasq”] ~

And that’s all there is to it. The next time materia runs on viscous it will automatically install and template the dnsmasq.container,dnsmasq.conf, hosts, and ethers files onto the host and start the dnsmasq container.