Linux, macOS, FreeBSD and Windows agent
A small binary installed on the machine to monitor. It measures CPU, memory, disks, network, services and containers — and, where the machine exposes them, temperatures, disk health, ZFS pools and KVM virtual machines — then pushes its readings to the DumbMonit server over HTTPS, on an encrypted channel of its own (see Agent channel security).
Network ports to open
Section titled “Network ports to open”From the agent to the DumbMonit server:
| Port | Protocol | Purpose |
|---|---|---|
| 8443 | TCP | Readings and commands: the encrypted agent channel (TLS 1.3 and a client certificate) |
| 8080 | TCP | Install script and agent download, at install and update time (HTTPS; 443 behind a reverse proxy) |
The direction is deliberate: the agent calls the server, never the other way round. It crosses NAT and home firewalls, and requires opening no port on the monitored machine.
What it watches
Section titled “What it watches”All metrics are prefixed dumbmonit_. Identity labels (target, host,
tag_*) are set by the server on reception, never by the agent: a machine
cannot write into another machine’s series, even by forging its labels.
| Family | Metrics | Labels |
|---|---|---|
| CPU | cpu_usage_percent, cpu_count, load_average_1/5/15; cpu_core_usage_percent only with cpu_per_core: true | core |
| Memory | memory_total/used/available_bytes, memory_used_percent, swap_total/used_bytes, swap_used_percent | |
| Filesystems | filesystem_total/used/free_bytes, filesystem_used_percent | mountpoint, device, fstype |
| Network (counters) | if_octets_in/out, if_packets_in/out, if_errors_in/out | ifname |
| Disks (counters) | disk_read_bytes, disk_written_bytes, one series per block device | device |
| Host | uptime_seconds, process_count | |
| Services | service_up (1 = running) | service |
| Containers | container_up, container_stop_expected (stopped containers only: 1 when Docker would not restart it — one-shot run finished, never started, stopped by hand without a restart policy), container_count, container_running_count, container_series_skipped (containers beyond docker_max_containers), container_cpu_percent (percent of one core, from the second cycle), container_memory_bytes (cache excluded), container_memory_limit_bytes and container_memory_percent (only with a limit), container_net_rx_bytes / container_net_tx_bytes and container_block_read_bytes / container_block_write_bytes (cumulative counters), container_pids, and the host-wide container_df_images_bytes, container_df_containers_bytes, container_df_volumes_bytes, container_df_build_cache_bytes, container_df_reclaimable_bytes (read every ten minutes) | container, image |
| Sensors | agent_sensor_temperature_celsius, agent_sensor_temperature_critical_celsius (the threshold the hardware itself declares), agent_sensor_fan_rpm | sensor, fan |
| Disk health (SMART) | agent_disk_smart_ok (1 = the disk passes its own self-assessment), agent_disk_temperature_celsius, agent_disk_power_on_hours, agent_disk_wearout_percent, agent_disk_reallocated_sectors, agent_disk_pending_sectors, agent_disk_media_errors | device, model |
| KVM / libvirt | agent_libvirt_domains, agent_libvirt_domains_running, agent_libvirt_pools; per domain agent_libvirt_domain_state_info, agent_libvirt_domain_running, agent_libvirt_domain_crashed, agent_libvirt_domain_vcpus, agent_libvirt_domain_memory_max_bytes, agent_libvirt_domain_memory_used_bytes; per storage pool agent_libvirt_pool_state_info, agent_libvirt_pool_active, agent_libvirt_pool_capacity_bytes, agent_libvirt_pool_allocation_bytes, agent_libvirt_pool_used_percent — see KVM virtual machines through libvirt | domain, pool, state |
| ZFS | agent_zfs_pool_health (0 online, 1 degraded, 2 worse), agent_zfs_pool_size/used/free_bytes, agent_zfs_pool_used_percent, agent_zfs_pool_fragmentation_percent, agent_zfs_pool_device_errors, agent_zfs_pool_data_errors, agent_zfs_pool_scrub_running, agent_zfs_pool_scrub_errors, agent_zfs_pool_scrub_age_seconds | pool |
| WireGuard | agent_wireguard_interface_peers; per peer agent_wireguard_peer_handshake_age_seconds, agent_wireguard_peer_has_handshake, agent_wireguard_peer_keepalive_seconds, agent_wireguard_peer_rx_bytes, agent_wireguard_peer_tx_bytes (counters) — see WireGuard tunnels | interface, peer, allowed_ips, name |
| IPsec (strongSwan) | agent_ipsec_tunnel_up; when up, agent_ipsec_tunnel_bytes_in/out (counters) and agent_ipsec_tunnel_rekey_seconds — see IPsec tunnels | connection, tunnel |
| Internet speed test | agent_speedtest_ok, agent_speedtest_age_seconds, agent_speedtest_download_bytes_per_second, agent_speedtest_upload_bytes_per_second, agent_speedtest_latency_ms, agent_speedtest_jitter_ms; the configured thresholds echoed as agent_speedtest_download_min_bytes_per_second, agent_speedtest_upload_min_bytes_per_second, agent_speedtest_max_latency_ms — see Internet speed test | |
| restic / Borg | agent_backup_repo_reachable, agent_backup_repo_last_snapshot_age_seconds, agent_backup_repo_last_snapshot_bytes (restic) — see restic and Borg backups | tool, repo |
| Windows performance counters | agent_perf_counter, one series per counter listed in perf_counters | counter |
| MDaemon (Windows) | mdaemon_queue_messages, mdaemon_queue_frozen, mdaemon_sessions_active, mdaemon_sessions_total, mdaemon_messages_total, mdaemon_filtered_messages_total, mdaemon_server_active, mdaemon_running, mdaemon_uptime_seconds — see MDaemon | queue, protocol, filter, verdict, server |
| Hyper-V (Windows) | hyperv_vms_health_ok, hyperv_vms_health_critical, hyperv_host_cpu_percent, hyperv_host_cpu_guest_percent, hyperv_partitions; per VM hyperv_vm_physical_memory_megabytes, hyperv_vm_guest_visible_memory_megabytes, hyperv_vm_memory_pressure_percent; per virtual disk hyperv_vhd_read_bytes_per_second, hyperv_vhd_write_bytes_per_second, hyperv_vhd_latency_seconds, hyperv_vhd_queue_length, hyperv_vhd_errors_total — see Hyper-V hosts | vm, disk |
| Agent | agent_collect_seconds, agent_buffered_samples, agent_dropped_samples |
A series that cannot be measured is absent, never zero: no smartctl, no
agent_disk_*; no readable probe, no agent_sensor_*; no ZFS, no
agent_zfs_*; no virsh, no agent_libvirt_*. A temperature outside 1–150 °C is dropped rather than published,
and a fan reading 0 rpm is dropped too — a motherboard has more fan headers
than fans, and an empty one reads zero like a dead fan does.
What is collected where
Section titled “What is collected where”| Linux | macOS | FreeBSD | Windows | |
|---|---|---|---|---|
| CPU, memory, swap, filesystems, network, uptime, processes | yes | yes | yes | yes |
| Load average | yes | yes | yes | no (Windows has no such notion) |
| Services | systemd (systemctl) | launchd (launchctl) | rc.d (service) | Windows services |
| Docker containers | yes | yes, if docker_socket points at Docker Desktop’s socket | no (no Docker daemon) | no |
| Temperatures | /sys/class/hwmon | SMC sensors | dev.cpu.N.temperature (CPU only) | no |
| Fans | /sys/class/hwmon | no | no | no |
| Disk health (SMART) | smartctl | smartctl (Homebrew) | smartctl (sysutils/smartmontools) | smartctl |
| ZFS pools | zpool (OpenZFS) | no | zpool | no |
| KVM virtual machines (libvirt) | virsh | no | no | no |
| OS health (updates, reboot, failed units, SELinux) | yes | no | no | no |
| Plakar backups | yes | yes | yes | yes |
| WireGuard tunnels | wg show | when wg is installed (not tested) | when wg is installed (not tested) | no |
| IPsec tunnels | swanctl | when swanctl is installed (not tested) | when swanctl is installed (not tested) | no |
| Internet speed test | yes, opt-in | yes, opt-in | yes, opt-in | yes, opt-in |
| restic and Borg backups | yes | yes | yes | restic only (not tested; Borg has no Windows build) |
| Performance counters, MDaemon queues, Hyper-V | no | no | no | yes |
| Machine identity | /etc/machine-id | IOPlatformUUID | kern.hostuuid, /etc/hostid | host name |
| Relay mode | yes | yes | yes | yes |
Windows has no temperature row on purpose: reading sensors there means going through WMI, with administrative rights, for values most machines do not expose at all. Publishing nothing is the honest answer.
smartctl (smartmontools), zpool and virsh are looked for at every
reading period and used when present; none is installed by the agent, and none
is required. All need root (virsh also works for a member of the libvirt
group) — the service the installer registers runs as root, a
hand-started agent under your own account will find nothing.
Counters are sent raw, so that a restart of the agent never produces a fake
rate; apply rate() in queries.
Built-in rules that apply: Monitor unreachable, High CPU, Unusual CPU. “Monitor unreachable” works on agents too: the server checks the age of the last batch received and declares the machine silent after three missed periods (at least 90 seconds).
Four more fire only on machines that produce the matching series, so they need no setting up and cannot go off on a machine without the hardware:
| Rule | Fires when |
|---|---|
| Disk SMART failing (critical) | a disk’s own self-assessment reports it as failing |
| ZFS pool degraded (critical) | a pool is no longer ONLINE for five minutes |
| ZFS scrub found errors (critical) | the last scrub found errors, or the pool reports permanently corrupted files |
| KVM domain crashed (critical) | libvirt reports a domain as crashed |
| libvirt pool almost full (warning) | a storage pool’s allocation reaches 90% of its capacity for fifteen minutes |
| Temperature above critical (critical) | a sensor exceeds the critical threshold its own hardware declares — no value is hard-coded, 85 °C being an emergency on a disk and a normal afternoon on a laptop CPU |
Several more watch what you declare or run yourself, and stay silent otherwise: WireGuard peer silent (a peer with a persistent keepalive and no handshake for 15 minutes), IPsec tunnel down (a configured tunnel with no installed session for ten minutes), restic/Borg backup too old (no snapshot for 48 hours), restic/Borg repository unreachable, and Internet download rate low / Internet upload rate low / Internet latency high (only once a threshold option is set) — see WireGuard tunnels, IPsec tunnels, restic and Borg backups and Internet speed test.
On a Windows server that runs MDaemon or SecurityGateway, Mail server Windows service stopped watches their services, and for MDaemon five more watch its queues: MDaemon mail queue growing, Mail queue stuck, MDaemon queue frozen, MDaemon Bad queue not empty and MDaemon Retry queue high (see Rules). On a Hyper-V host, three watch the virtual machines: Hyper-V VM health critical, Hyper-V host CPU high and Hyper-V virtual disk errors (see Hyper-V hosts).
Install
Section titled “Install”The steps below are the ones the notice next to the form shows. There is no account to create on the machine: the enrollment token is the only secret.
-
In DumbMonit, add a monitor and pick Server with the agent, then choose how the machine reaches the server: directly, or through the relay of a remote site behind a firewall. An enrollment token (its secret starts with dmon_; the token is listed as tok_… afterwards) is shown once, with the install command ready to copy for Linux/macOS, Windows and Docker. See Monitor a remote site for the relay.
-
Run that install command on the machine, with elevated rights (sudo on Linux and macOS, an elevated PowerShell on Windows).
-
The page waits for the first report (a few seconds), then lets you name the machine, file it in a folder and place it at a site, on the spot.
-
Optional, for Docker: if the agent runs under a dedicated user, add that user to the “docker” group and restart the agent.
usermod -aG docker dumbmonit -
For a fleet (a playbook, a machine image), tick Several machines with this command; a whole Windows domain goes through the Active Directory kit, offered on the same page. Revoke a lost token in Settings → Agents. See Active Directory.
Good to know
Section titled “Good to know”The token is the agent’s key to push its measurements to DumbMonit. It is not an
account on the machine. A token created from the monitor form is single use: it
enrols this one machine and nothing else. For a fleet, tick Several machines
with this command in the same flow. On that first batch the server gives the agent a secret of
its own and stores it in /etc/dumbmonit/agent-secret, readable by nobody else:
from then on, that machine is the only one that can report under this monitor.
Docker: to see the containers and let DumbMonit restart or update them, the agent must reach the Docker socket. The service the installer registers already can. Restart and auto-update policies are then set per container on the monitor page.
Plakar backups are detected automatically: a Plakar monitor appears under the
machine when the plakar binary or a kloset (~/.config/plakar/stores.yml of
every user, ~/.plakar, /var/lib/plakar) is found, and nothing is shown
otherwise. Set plakar_klosets in agent.yaml to watch a fixed list, or
plakar: false to opt out.
The services the agent finds (Docker, Plakar, restic/Borg, ZFS, libvirt, WireGuard, IPsec, speed test) each become their own monitor under the machine, with their own status and alerts. The machine’s “Services found by the agent” panel lists what was found and what was not: untick a module to drop its monitor, tick one to watch it before it shows up. See Services found by the agent.
Two things are worth reading before you roll this out to more than one machine: Binding, which is what makes a monitor really belong to one agent installation, and Single use or fleet, which is the choice you make when you create a token.
curl -sSL https://server:8080/install.sh | sh -s -- --token=dmon_xxx --url=https://server:8080The script is POSIX sh (BusyBox and dash work), downloads the binary for
the machine’s architecture (x86_64 or aarch64) from the server, writes
/etc/dumbmonit/agent.yaml (mode 0600) and registers dumbmonit-agent with
systemd, or with OpenRC on Alpine. The systemd unit is hardened
(ProtectSystem=strict, NoNewPrivileges, MemoryMax=256M,
CPUQuota=20%).
The same command as on Linux, run with sudo:
curl -sSL https://server:8080/install.sh | sudo sh -s -- --token=dmon_xxx --url=https://server:8080The macOS binaries are not in the DumbMonit image: building them
requires Apple’s SDK, which its licence forbids redistributing. They are
attached to each GitHub release
instead — dumbmonit-agent-macos-aarch64 (Apple silicon) and
dumbmonit-agent-macos-x86_64 (Intel), each with its .sha256 — and the
server redirects /download/dumbmonit-agent-macos-… there, so the script
picks the one for the Mac’s architecture (uname -m). The server does not
ship those binaries, so the install command carries no checksum for them:
add --sha256=macos-aarch64:<hex> (or macos-x86_64) with the value of the
release’s .sha256, or the installer refuses the download. The Mac also
needs to reach github.com.
Without that access, or to install a binary you already have, download it
and pass it with --bin (compare it with the release’s .sha256 first:
--bin installs what you give it):
curl -sSLO https://github.com/laupernoe/dumbmonit/releases/latest/download/dumbmonit-agent-macos-aarch64curl -sSL https://server:8080/install.sh | sudo sh -s -- \ --token=dmon_xxx --url=https://server:8080 --bin=./dumbmonit-agent-macos-aarch64The script installs the binary in /usr/local/bin (and removes the
quarantine attribute a browser download carries, which would make
Gatekeeper refuse to start it), writes
/usr/local/etc/dumbmonit/agent.yaml (mode 0600) and registers the system
daemon com.dumbmonit.agent in /Library/LaunchDaemons, started with
launchctl bootstrap system. It runs at boot, with no one logged in.
fetch -qo - https://server:8080/install.sh | sh -s -- --token=dmon_xxx --url=https://server:8080x86_64 only. The script downloads dumbmonit-agent-freebsd-x86_64 from the
server, writes /usr/local/etc/dumbmonit/agent.yaml (mode 0600) and
installs /usr/local/etc/rc.d/dumbmonit_agent, enabled with
service dumbmonit_agent enable. daemon(8) supervises the agent,
restarts it if it stops and writes /var/log/dumbmonit-agent.log.
Works the same on TrueNAS CORE and other FreeBSD-based appliances, where the ZFS series are the point of the exercise.
& ([scriptblock]::Create((irm https://server:8080/install.ps1))) -Token dmon_xxx -Url https://server:8080The script downloads dumbmonit-agent-windows-x86_64.exe from the server,
checks it against its SHA-256, installs it as
C:\Program Files\DumbMonit\dumbmonit-agent.exe, writes
C:\ProgramData\DumbMonit\agent.yaml (readable by SYSTEM and
Administrators only) and registers the Windows service DumbMonitAgent
(“DumbMonit system agent”): started automatically at boot, as LocalSystem,
and restarted after a failure (10 s, 30 s, then 60 s). The agent talks to
the service manager itself (--service), so stopping the service lets it
send what it still holds before exiting.
A service has no console: the agent writes its log to
C:\ProgramData\DumbMonit\agent.log, set aside as agent.log.old past
10 MB.
There is no native ARM64 build: on Windows 11 for ARM, the script installs the x86_64 binary, which runs under x64 emulation.
For a whole Windows domain, deploy the agent with one group policy instead of running the command on each server: see Deploy the agent with Active Directory.
One token can enrol several machines: name it after a group or a machine.
Native binaries
Section titled “Native binaries”The agent is a single static executable per system, with nothing to install next to it:
| System | File | Where to get it |
|---|---|---|
| Linux x86_64 | dumbmonit-agent-linux-x86_64 | the server, /download/… |
| Linux aarch64 (64-bit ARM: Raspberry Pi OS 64-bit, ARM servers) | dumbmonit-agent-linux-aarch64 | the server, /download/… |
| Windows x86_64 | dumbmonit-agent-windows-x86_64.exe | the server, /download/… |
| FreeBSD x86_64 | dumbmonit-agent-freebsd-x86_64 | the server, /download/… |
| macOS Apple silicon | dumbmonit-agent-macos-aarch64 | the GitHub release (the server redirects there) |
| macOS Intel | dumbmonit-agent-macos-x86_64 | the GitHub release (the server redirects there) |
The Linux binaries are linked statically against musl: they run on any
distribution, glibc or not (Alpine, NAS firmwares), with no library to match.
Each file has its checksum next to it, <file>.sha256, in sha256sum
format. Every GitHub release
carries all six binaries and their checksums, for machines that cannot reach
the server’s /download/ or for installing with --bin.
Since 0.9.0, every binary of a release is also signed keylessly with
Sigstore (<file>.sigstore.json) and comes with
a SLSA provenance attestation. To check that a binary was built by this
repository’s release workflow:
cosign verify-blob dumbmonit-agent-linux-x86_64 \ --bundle dumbmonit-agent-linux-x86_64.sigstore.json \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ --certificate-identity-regexp '^https://github.com/laupernoe/dumbmonit/\.github/workflows/release\.yml@refs/tags/v'# or, with the GitHub CLI:gh attestation verify dumbmonit-agent-linux-x86_64 -R laupernoe/dumbmonitTo build the Linux, Windows and FreeBSD binaries yourself, from a clone of the repository (Docker is the only requirement):
docker buildx build --target agent-dist --output type=local,dest=dist/agent .Managing the service
Section titled “Managing the service”| Linux (systemd) | Linux (OpenRC) | macOS (launchd) | FreeBSD (rc.d) | Windows (PowerShell) | |
|---|---|---|---|---|---|
| Status | systemctl status dumbmonit-agent | rc-service dumbmonit-agent status | sudo launchctl print system/com.dumbmonit.agent | service dumbmonit_agent status | Get-Service DumbMonitAgent |
| Logs | journalctl -u dumbmonit-agent -f | tail -f /var/log/dumbmonit-agent.log | tail -f /var/log/dumbmonit-agent.log | tail -f /var/log/dumbmonit-agent.log | Get-Content C:\ProgramData\DumbMonit\agent.log -Tail 20 -Wait |
| Restart | systemctl restart dumbmonit-agent | rc-service dumbmonit-agent restart | sudo launchctl kickstart -k system/com.dumbmonit.agent | service dumbmonit_agent restart | Restart-Service DumbMonitAgent |
| Stop | systemctl stop dumbmonit-agent | rc-service dumbmonit-agent stop | sudo launchctl bootout system/com.dumbmonit.agent | service dumbmonit_agent onestop | Stop-Service DumbMonitAgent |
| Configuration | /etc/dumbmonit/agent.yaml | /etc/dumbmonit/agent.yaml | /usr/local/etc/dumbmonit/agent.yaml | /usr/local/etc/dumbmonit/agent.yaml | C:\ProgramData\DumbMonit\agent.yaml |
| Service file | /etc/systemd/system/dumbmonit-agent.service | /etc/init.d/dumbmonit-agent | /Library/LaunchDaemons/com.dumbmonit.agent.plist | /usr/local/etc/rc.d/dumbmonit_agent | service DumbMonitAgent (services.msc) |
--uninstall (-Uninstall on Windows) removes all of it, service included,
on every system.
Removing an agent
Section titled “Removing an agent”Deleting a machine’s monitor does not uninstall its agent: the agent keeps running and asks to enrol again. When you delete the monitor, DumbMonit offers Remove the agent automatically if the agent can do it:
- it accepts commands (
commands: true, the default; an agent configured withcommands: falsenever receives the request); - it reported in the last few minutes;
- it was installed by
install.shorinstall.ps1, outside a container.
The agent then starts a helper that, a few seconds later, does what
--uninstall (-Uninstall) does: it stops and removes the service and deletes
the binary and the configuration. Nothing is downloaded. The agent confirms
just before the clean-up and stops; DumbMonit then deletes the monitor. Only an
administrator signed in to the web interface can ask for it (API tokens
cannot), and the request is recorded in the audit log.
An agent running in Docker refuses: remove its container instead
(docker rm -f dumbmonit-agent, then docker volume rm dumbmonit-agent). If
the agent refuses, does not answer within two minutes or is offline, the page
shows the commands to run on the machine, and Delete the monitor anyway.
By hand, at any time, the install command with --uninstall (-Uninstall on
Windows) in place of its other flags does the same.
Installer flags
Section titled “Installer flags”install.sh | install.ps1 | Effect |
|---|---|---|
--token=TOKEN | -Token | Enrollment token (required). |
--url=URL | -Url | Server URL, for example https://server:8080. On Linux, DUMBMONIT_URL in the environment is used if the flag is absent. |
--interval=N | -Interval | Sampling period in seconds (default 30). |
--services=a,b,c | -Services @('a','b') | Services whose state is reported, named as the host system names them: systemd units on Linux, launchd labels (com.apple.sshd) on macOS, rc.d names on FreeBSD, Windows services. |
--tags=key=value,… | -Tags @{key='value'} | Tags, copied as tag_<key> on every series. |
--pinned-key=sha256//… | -PinnedKey | Public key of the server’s self-signed certificate. The download of the script and of the binary is accepted from that key only, and the agent keeps it. Included in the command DumbMonit shows when the server uses its own certificate. |
--tls-url=URL | -TlsUrl | Address of the encrypted agent channel (default: the host of --url, port 8443). |
--ca-pin=SHA256 | -CaPin | Fingerprint of the server’s certificate authority: the agent then only talks to that server, never in clear. |
--hostname=NAME | -HostName | Name announced to the server (default: the machine’s). |
--modules=a,b | -Modules a,b | Optional modules to run — see Choosing the modules. Without it, every module detects itself; none keeps system metrics only. The install command shown by DumbMonit adds it when you tick modules. |
--plakar-klosets=a,b | Plakar klosets to read (@name or a path); default: every kloset found. | |
--bin=PATH | -BinPath | Local binary to install instead of downloading it. |
--sha256=LIST | -Sha256 | Expected SHA-256 of the download, as platform-arch:hex pairs separated by commas (a bare hex value also works). Included in the command DumbMonit shows; required unless the next flag is given. |
--insecure-skip-checksum | -InsecureSkipChecksum | Install a downloaded binary without verifying it, with a warning. Only for air-gapped or manual setups. |
--no-start | Install everything, but do not contact the server or start the service (machine image, testing). | |
--uninstall | -Uninstall | Uninstall the agent and delete its configuration. |
--help | Show the help. |
Upgrade
Section titled “Upgrade”From this version on, agents update themselves (see
Automatic updates): after upgrading the server there is
nothing to run. The install command below is still the way to do the first
update of an agent that predates this feature, and to change its token or URL.
Take the command from the current server; the binary now lives in
/usr/local/lib/dumbmonit-agent/ on Linux.
Both scripts are idempotent: running the install command again, with the same
token and URL, replaces the binary, rewrites the configuration and restarts the
service. That is the upgrade procedure. Take the command from the current
server (add a monitor, Server with the agent): its --sha256 must match the binaries of the
server version you upgraded to.
The same command migrates a machine still running the EzyMonit agent, on Linux
and on Windows. The installer stops and removes the old service
(ezymonit-agent, or EzyMonitAgent on Windows), moves the configuration to
the new path (/etc/dumbmonit/agent.yaml, C:\ProgramData\DumbMonit\agent.yaml)
and installs dumbmonit-agent in its place. The old ezym_… token stays
valid, so the command shown when the token was created still works; nothing
has to be revoked first.
Automatic updates
Section titled “Automatic updates”From this version on, the agent updates itself from your server: after you upgrade DumbMonit, the agents follow without re-running any install command. The first update of an agent that predates this feature is manual (see Agents installed before this version).
How it works
Section titled “How it works”- On every batch the agent reports the SHA-256 of its running binary, its
platform (
linux-x86_64,linux-aarch64,freebsd-x86_64,windows-x86_64), whether it is able to replace itself, and the outcome of its last update attempt. - The server ships the agent binaries of its own build (
/agentsin the image,DUMBMONIT_AGENT_DIR). When the agent’s SHA-256 differs from the server’s binary for its platform, and updates are allowed for that agent, the server’s reply to the batch carries an offer: version, build, file, SHA-256, size and Ed25519 signature. The offer is made on the encrypted agent channel (port 8443) and on the old plain channel. - The build decides, not only the version: a
:edgeserver aligns its agents on its own build. An agent is never downgraded, unless an administrator asks for it: Update now on an agent newer than the server offers Install the server’s version. - The agent downloads the file with
GET /api/agent/update/download/<file>through its own channel (encrypted, pinned, mutual TLS when it has a certificate), presenting the enrolment token, the binding secret and the machine key. A neighbour holding the fleet token cannot fetch it as another machine. - Before anything is replaced, the agent checks that the size is at most
128 MiB, that the SHA-256 equals the offer, that the Ed25519 signature
verifies against a release public key compiled into the agent
(see Security), and then runs the new binary
once with
--version: it must print the offered version.
Linux and FreeBSD. The old binary is copied next to the new one
(dumbmonit-agent.previous), an update ledger (dumbmonit-agent.update.json)
is written, then the new binary replaces the old one with an atomic, fsync’d
rename and the agent exec()s it in the same process: systemd, OpenRC and
rc.d see the same PID. Measurements still buffered are flushed, or kept on
disk, first.
Windows. A running .exe cannot be overwritten. The agent copies itself to
dumbmonit-agent-update-helper.exe, starts it detached with --update-helper
and stops. The helper waits for the DumbMonitAgent service to stop, swaps
dumbmonit-agent.exe and the staged dumbmonit-agent.new.exe (the old one is
kept as .previous), starts the service and watches the trial.
On Linux the install script puts the binary in
/usr/local/lib/dumbmonit-agent/dumbmonit-agent; a symlink stays at
/usr/local/bin/dumbmonit-agent. The systemd unit gets
ReadWritePaths=/usr/local/lib/dumbmonit-agent: with ProtectSystem=strict,
that folder is the only place the agent can write besides its cache.
Settings
Section titled “Settings”Settings → Agents → Automatic agent updates (admin session only; API tokens
cannot change these settings). The same settings are available through
GET/PUT /api/agent/updates, PUT /api/agent/updates/{id},
POST /api/agent/updates/{id}/now and POST /api/agent/updates/all.
| Setting | Meaning |
|---|---|
| Agents update themselves | On by default, for new and existing installations. Each agent can override it: follow the global setting, on, or off. |
| Window | Any time (default), only during a maintenance window (an active silence on the device) or manual only (Update now). |
| Progressive rollout | Agents marked canary get the update first. The others wait until every eligible canary has run the new build for N minutes (default 60, maximum 10080) without failure; a failed canary holds the rollout. With no canary, everyone updates at once. |
| Update now / Update all now | Per agent, or for all of them: ignores the window and the rollout, and retries a failed build once. |
| Accept unsigned builds (development) | Off by default, with a warning in the UI. See Security. |
The agent list shows the version and an update status for each agent: up to date, pending, updating, failed (with the reason), refused, disabled, manual, waiting for window, waiting for canaries, newer than server, legacy, container, read-only, disabled locally, unsupported, no binary.
Security
Section titled “Security”- Integrity: the SHA-256 comes from the server.
- Authenticity: an Ed25519 detached signature, made with the release key
held only in the GitHub secret
AGENT_SIGNING_KEY. The public key (several are allowed, for rotation) is compiled into the agent (crates/agent/signing/release-keys.pem). A signature that does not verify is always refused, whatever the settings. - Unsigned builds (no
.sig: local builds, forks, CI without the secret) are refused by default. They are accepted only when both the server setting Accept unsigned builds (development) is on and the offer arrived over the encrypted agent channel, where the server is authenticated by the CA pin the agent got from the install command of that same server. On the plain channel anyone in the middle could set the flag, so signatures are mandatory there. A refusal is reported (status refused) without downloading anything, and is evaluated again when the setting changes. - Signatures are separate files (
<binary>.sig, 64 raw bytes, also served at/download/<binary>.sig), so the binaries stay bit-for-bit reproducible.
To verify a binary by hand:
openssl pkeyutl -verify -pubin -inkey release-key.pub.pem -rawin \ -in dumbmonit-agent-linux-x86_64 -sigfile dumbmonit-agent-linux-x86_64.sigHow the keys are made and used by the release workflow: Signing the agent binaries. The agent channel itself is described in Agent channel security.
Rollback
Section titled “Rollback”The new binary is on trial until the server accepts its first batch. If that
does not happen within 120 s (150 s for the Windows helper), or if the binary
stops at startup more than 3 times (a crash loop under systemd
Restart=always, or the Windows recovery actions), the previous binary is
restored and restarted. The failure (rolled_back and its reason) is reported
to the server with the next batch.
That build is then not retried automatically on that machine, neither by the agent nor by the server, until an administrator clicks Update now, which retries once. OpenRC without supervision does not restart a crashed agent: the rollback then happens at its next start.
Docker and other cases
Section titled “Docker and other cases”- Docker: an agent in a container never replaces itself (the image
ghcr.io/laupernoe/dumbmonit-agentsetsDUMBMONIT_AGENT_CONTAINER=1;/.dockerenvand/run/.containerenvare also detected). The UI says to update its image:docker compose pull && docker compose up -d. - macOS: the server image does not ship macOS agents. Update them with
install.sh(status unsupported). - Agents installed before this version do not know how to update themselves (status legacy): re-run the install command once.
- Folder not writable: an agent that cannot write to its folder reports read-only. Re-run the install command once (it sets up the new layout).
Turning it off
Section titled “Turning it off”- For everyone: turn off Agents update themselves in Settings → Agents, or choose manual only to update on demand.
- For one agent: set its override to off.
- On the machine itself:
self_update: falseinagent.yaml, orDUMBMONIT_AGENT_SELF_UPDATE=false(status disabled locally). It always wins over the server.
Services found by the agent
Section titled “Services found by the agent”The machine’s own monitor shows the machine: CPU, memory, filesystems, network,
disk health (SMART), sensors, OS health and the services you list in
services. Everything else the agent detects is a service, and each
service it finds becomes a monitor of its own, of kind agent_service, filed
under the machine:
| Service | Its metrics | Monitor name |
|---|---|---|
| Docker | container_* | <machine> · Docker |
| Plakar | backup_* | <machine> · Plakar |
| restic / Borg | agent_backup_repo_* | <machine> · restic/Borg |
| ZFS | agent_zfs_* | <machine> · ZFS |
| KVM / libvirt | agent_libvirt_* | <machine> · libvirt |
| WireGuard | agent_wireguard_* | <machine> · WireGuard |
| IPsec | agent_ipsec_* | <machine> · IPsec |
| Internet speed test | agent_speedtest_* | <machine> · Speed test |
The split happens on the server, by metric name, so it works with every agent
version. Each series is written once, with target set to the monitor it
belongs to: the service’s series carry the service monitor’s id, the machine’s
series the machine’s id. Nothing is counted twice.
A service monitor has its own status, alerts, rules, rule overrides and
silences. It is unreachable when the agent stops reporting it while the
machine still reports (the Docker daemon stopped, wg removed…), after the
same three missed periods as the machine itself. When the machine goes silent,
its services are shown as hidden behind it (suppressed) instead of raising
alerts of their own. A new service monitor starts with the machine’s tags, so
tag-based silences and rules keep applying.
There is nothing to choose before the install. After the first report, the machine’s page lists every module in Services found by the agent:
- a found service is ticked and links to its monitor;
- a service not found is unticked; tick it to create its monitor anyway (it stays unreachable until the agent reports it — useful to be told when a service you rely on is missing);
- untick a found service to delete its monitor and drop its readings. The server tells the agent in its next acknowledgement, and the agent stops collecting it (an older agent keeps collecting, and the server keeps dropping). Deleting a service monitor has the same effect. Tick it again to bring it back.
Deleting the machine deletes its service monitors. Container actions and policies work from the Docker monitor page; the commands still go to the machine’s agent.
The same as the monitor notice:
- Install the agent on the machine: each service it finds becomes a monitor under that machine on its first report.
- To stop watching one, untick it in the machine’s “Services found by the agent” panel or delete its monitor.
Once a monitor is deleted, the agent’s readings for that service are dropped.
Upgrading an existing installation
Section titled “Upgrading an existing installation”Nothing to do. The first time the agent reports a service after the upgrade, the server creates its monitor and moves onto it what concerned that service on the machine:
- rule overrides of the rules that read the service’s metrics are copied to the service monitor;
- rules limited to the machine (
idsselector) that read the service’s metrics now also select the service monitor; - the machine’s silences (maintenance windows) are copied to the service monitor;
- alerts in progress on the service’s series are moved to the service monitor under their new fingerprint, so they do not resolve and fire again.
The history stays where it was written: VictoriaMetrics cannot relabel existing series, so readings from before the upgrade remain under the machine (they age out with the retention), and the service monitor’s charts start at the upgrade. Anomaly baselines of the moved series start learning again. The alert history (resolved alerts) also stays with the machine.
Choosing the modules
Section titled “Choosing the modules”By default every optional module detects itself and reports what it finds: Docker, Plakar, system health (pending updates, reboot required, failed units, SELinux), sensors, SMART, ZFS, KVM (libvirt), WireGuard, IPsec and, on Windows, the MDaemon and Hyper-V counter sets. To watch only some of them, list them:
modules: [docker, plakar]or pick them in the enrolment screen (Choose the modules, an advanced
option: by default the install command asks nothing), which adds
--modules=docker,plakar to the install command. Only the listed modules run;
the others send no series, so they get no monitor and raise no alert. To stop
watching a service on a machine already enrolled, untick it in
Services found by the agent instead: no need to
touch agent.yaml. CPU,
memory, filesystems, network and the machine’s OS identity are always
collected. modules: [] (or --modules=none) keeps those only.
Known names: docker, plakar, system_health, sensors, smart, zfs,
libvirt, wireguard, ipsec, perf_counters. An unknown name stops the agent at start-up with
the list, rather than silently switching a module off.
DUMBMONIT_AGENT_MODULES=docker,plakar overrides the file (an empty value is
ignored). An explicit flag (smart: true, DUMBMONIT_AGENT_SMART=1) still
wins over the list. The agent logs the modules it runs at start-up.
Configuration file
Section titled “Configuration file”/etc/dumbmonit/agent.yaml on Linux, /usr/local/etc/dumbmonit/agent.yaml on
macOS and FreeBSD, C:\ProgramData\DumbMonit\agent.yaml on Windows:
server_url: https://server:8080token: dmon_...interval_secs: 30 # sampling periodhostname: nas-basement # optional: name announced to the serverservices: # systemd units or Windows services - sshd - dockertags: # free labels, prefixed "tag_" on the server role: nas room: basementdocker: true # container inventory (default: true)docker_socket: /var/run/docker.socksensors: true # temperatures and fans (default: true)smart: true # disk health through smartctl (default: true)smart_bin: smartctl # path, when it is not in the service's PATHsmart_interval_secs: 300 # minimum 60: a sleeping disk is never woken to be readzfs: true # ZFS pools through zpool (default: true)zfs_bin: zpoolzfs_interval_secs: 120 # minimum 30libvirt: true # KVM domains and storage pools through virsh (default: true)libvirt_bin: virshlibvirt_uri: qemu:///systemlibvirt_interval_secs: 60 # minimum 15wireguard: true # WireGuard tunnels through wg (default: true), see belowipsec: true # IPsec tunnels through swanctl (default: true), see belowspeedtest: false # Internet speed test (default: false, opt-in), see belowspeedtest_download_url: https://speed.example.lan/backend/garbage.php?ckSize=50speedtest_upload_url: https://speed.example.lan/backend/empty.phpinventory: true # daily list of installed packages, for the CVE scan (default: true)restic_repos: [] # restic repositories to watch, see "restic and Borg backups"borg_repos: [] # Borg repositories to watchmdaemon: auto # Windows: MDaemon queues when its service exists (true / false / auto)hyperv: auto # Windows: Hyper-V counters when the vmms service exists (true / false / auto)perf_counters: # Windows: performance counters to publish - '\Memory\Available MBytes' - path: '\Processor(_Total)\% Processor Time' name: cpu_total # value of the "counter" label; the path when omittedmax_buffered_samples: 20000log_level: infosensors, smart, zfs and libvirt are detected, never assumed: left at
true on a machine with no probe, no smartctl, no pool or no virsh, they
cost one look per period and emit nothing at all. Set one to false to stop even looking — worth doing
on a NAS whose disks you would rather smartctl never touched, although the
agent already passes -n standby so that a sleeping disk is measured only when
it is awake for other reasons.
KVM virtual machines through libvirt
Section titled “KVM virtual machines through libvirt”On a KVM hypervisor (a plain Linux host with libvirt, not Proxmox VE, which has
its own integration), install the agent on the host itself, not
in a guest. The agent runs virsh -c qemu:///system with list --all --name,
dominfo --domain <name>, pool-list --all --name and pool-info --pool <name>: it never defines, starts, stops or migrates a domain, and never
changes a storage pool. The explicit qemu:///system URI matters for an agent
run by a member of the libvirt group rather than root: without it, virsh
would read that user’s private, empty session. Set libvirt_uri to read
another connection.
agent_libvirt_domain_memory_used_bytes is absent for a domain that is shut
off: libvirt only knows its configured maximum. Storage pool sizes come from
virsh’s human-readable output, rounded to two decimals, so
agent_libvirt_pool_used_percent is an approximation, close enough to warn
before a pool fills up. At most 200 domains and 64 pools are detailed.
Next to it, /etc/dumbmonit/agent-secret (C:\ProgramData\DumbMonit\agent-secret
on Windows) holds the binding secret the server handed this machine. The agent
writes it with mode 0600 and rereads it at every start; it is not part of
agent.yaml because it is not something you set — see
Binding. DUMBMONIT_AGENT_SECRET_FILE moves
it, which is what you want for an agent in a container.
Each key can be overridden by the environment, which makes the agent usable in a container without mounting a file:
| Variable | Effect |
|---|---|
DUMBMONIT_AGENT_CONFIG | Path of the configuration file |
DUMBMONIT_AGENT_URL | Server URL |
DUMBMONIT_AGENT_TOKEN | Enrollment token |
DUMBMONIT_AGENT_SECRET_FILE | Where to keep the binding secret (default: agent-secret, next to the configuration file) |
DUMBMONIT_AGENT_INTERVAL_SECS | Sampling period |
DUMBMONIT_AGENT_HOSTNAME | Name announced to the server |
DUMBMONIT_AGENT_SERVICES | Services to watch, comma-separated |
DUMBMONIT_AGENT_TAGS | key=value, comma-separated |
DUMBMONIT_AGENT_DOCKER | true / false |
DUMBMONIT_AGENT_DOCKER_SOCKET | Docker socket path |
DUMBMONIT_AGENT_DOCKER_MAX_CONTAINERS | Containers detailed per host, default 200 (0: counts only) |
DUMBMONIT_AGENT_INTERFACES_IGNORE | Interfaces left out, comma-separated names or regexes; default `^(veth |
DUMBMONIT_AGENT_INTERFACES_ONLY | Interfaces to keep; when set, replaces the ignore list |
DUMBMONIT_AGENT_MOUNTS_IGNORE | Mount points left out of filesystems and disk I/O; default skips /var/lib/docker/, /run/…, /sys/, /proc/, /dev/, /snap/ |
DUMBMONIT_AGENT_CPU_PER_CORE | true to also send one CPU series per core (default false) |
DUMBMONIT_AGENT_SENSORS | true / false: temperatures and fans (default true) |
DUMBMONIT_AGENT_SMART | true / false: disk health through smartctl (default true) |
DUMBMONIT_AGENT_SMART_BIN | Path of smartctl |
DUMBMONIT_AGENT_SMART_INTERVAL_SECS | Period between two SMART readings, default 300, minimum 60 |
DUMBMONIT_AGENT_ZFS | true / false: ZFS pools through zpool (default true) |
DUMBMONIT_AGENT_ZFS_BIN | Path of zpool |
DUMBMONIT_AGENT_ZFS_INTERVAL_SECS | Period between two ZFS readings, default 120, minimum 30 |
DUMBMONIT_AGENT_LIBVIRT | true / false: KVM domains and storage pools through virsh (default true) |
DUMBMONIT_AGENT_LIBVIRT_BIN | Path of virsh |
DUMBMONIT_AGENT_LIBVIRT_URI | libvirt connection URI, default qemu:///system |
DUMBMONIT_AGENT_LIBVIRT_INTERVAL_SECS | Period between two libvirt readings, default 60, minimum 15 |
DUMBMONIT_AGENT_WIREGUARD | true / false: WireGuard tunnels through wg (default true) |
DUMBMONIT_AGENT_WIREGUARD_BIN | Path of wg |
DUMBMONIT_AGENT_IPSEC | true / false: IPsec tunnels through swanctl (default true) |
DUMBMONIT_AGENT_IPSEC_BIN | Path of swanctl |
DUMBMONIT_AGENT_SPEEDTEST | true / false: run the Internet speed test (default false) |
DUMBMONIT_AGENT_SPEEDTEST_DOWNLOAD_URL | Address downloaded to measure the download rate |
DUMBMONIT_AGENT_SPEEDTEST_UPLOAD_URL | Address posted to, to measure the upload rate |
DUMBMONIT_AGENT_SPEEDTEST_DOWNLOAD_BYTES_CAP | Bytes read at most per test, default 50000000 |
DUMBMONIT_AGENT_SPEEDTEST_UPLOAD_BYTES | Bytes sent per test, default 10000000 |
DUMBMONIT_AGENT_SPEEDTEST_INTERVAL_SECS | Period between two tests, default 21600 (6 h), minimum 300 |
DUMBMONIT_AGENT_SPEEDTEST_MIN_DOWNLOAD_MBPS | Alert threshold, download rate in Mbps (unset: no alert) |
DUMBMONIT_AGENT_SPEEDTEST_MIN_UPLOAD_MBPS | Alert threshold, upload rate in Mbps (unset: no alert) |
DUMBMONIT_AGENT_SPEEDTEST_MAX_LATENCY_MS | Alert threshold, latency in milliseconds (unset: no alert) |
DUMBMONIT_AGENT_RESTIC_BIN | Path of restic |
DUMBMONIT_AGENT_BORG_BIN | Path of borg |
DUMBMONIT_AGENT_BACKUP_REPOS_INTERVAL_SECS | Period between two readings of the restic and Borg repositories, default 600, minimum 60 |
DUMBMONIT_AGENT_MDAEMON | true, false or auto: MDaemon’s performance counters (default auto, read when the MDaemon Windows service exists) |
DUMBMONIT_AGENT_HYPERV | true, false or auto: Hyper-V’s performance counters (default auto, read when the vmms Windows service exists) |
DUMBMONIT_AGENT_PERF_COUNTERS | Windows performance counter paths, separated by semicolons (a comma can be part of an instance name); replaces perf_counters |
DUMBMONIT_AGENT_SYSTEM_HEALTH | true / false: operating-system health, the master switch of the next four (default true) |
DUMBMONIT_AGENT_CHECK_UPDATES | true / false: pending updates (default true) |
DUMBMONIT_AGENT_UPDATES_INTERVAL_SECS | Period between two pending-updates checks, default 3600, minimum 300 |
DUMBMONIT_AGENT_CHECK_REBOOT | true / false: reboot required (default true) |
DUMBMONIT_AGENT_CHECK_FAILED_UNITS | true / false: failed systemd units (default true) |
DUMBMONIT_AGENT_MAX_BUFFERED_SAMPLES | Size of the catch-up buffer |
DUMBMONIT_AGENT_LOG | trace, debug, info, warn, error |
DUMBMONIT_AGENT_RELAY | true to run, for the server, the probes of the monitors assigned to this agent (see Run the agent in Docker / on another network) |
DUMBMONIT_AGENT_SITE | Site label shown next to the agent when it is offered as a relay |
DUMBMONIT_AGENT_GATEWAY_LISTEN | A relay’s gateway for the agents of its site, for example 0.0.0.0:8443 (gateway_listen:); wins over the port set in the UI (see Install agents behind the relay) |
DUMBMONIT_AGENT_GATEWAY_MAX_CONNECTIONS | Connections the gateway holds at once, 64 by default (gateway_max_connections:) |
DUMBMONIT_AGENT_GATEWAY_RATE_KIB | Rate of each tunnel through the gateway, per direction, in KiB/s, 4096 by default (gateway_rate_kib:) |
Windows performance counters
Section titled “Windows performance counters”On Windows, the agent reads performance counters — the values Performance
Monitor (perfmon) shows — through the PDH library, and publishes them every
sampling period. Some software publishes its state nowhere else.
MDaemon. When the Windows service named MDaemon exists, the agent reads
the MDaemon performance object: queue sizes, sessions, message and spam
totals, the state of MDaemon’s internal servers and its uptime. The series,
the rules and the page that shows them are described on the
MDaemon page. mdaemon: true
reads them even without the service (a renamed service), mdaemon: false
never does. The presence of the service is checked again every ten periods,
so an MDaemon installed after the agent is picked up without a restart.
Mail servers (MDaemon and SecurityGateway). On the same detection, the
agent also reads, with no setting, what only the server itself can tell: the
state and start mode of the product’s Windows services, the version of its
executable, the number of messages and the age of the oldest one in its queue
folders, and the free space of the volume that holds its data. It asks
Windows through one PowerShell call per period (every tenth period while no
product is found). The series are agent_mail_* and are described on the
MDaemon and
SecurityGateway pages, which also say
how a monitor is linked to its agent. mdaemon: false switches this off too.
Hyper-V. When the Hyper-V Virtual Machine Management service (vmms)
exists, the agent reads Hyper-V’s performance objects; see
Hyper-V hosts. hyperv: true and hyperv: false force the
choice, as for MDaemon.
Any other counter. List its full path under perf_counters, as
\Object(Instance)\Counter or \Object\Counter for an object with a single
instance, and it is published as dumbmonit_agent_perf_counter{counter="…"},
with the name you give or the path itself:
perf_counters: - '\LogicalDisk(C:)\% Free Space' - path: '\MSExchangeTransport Queues(_total)\Messages Queued For Delivery' name: exchange_queue- Write the counter names in English, as they appear in Performance
Monitor on an English Windows: the agent adds them with
PdhAddEnglishCounterW, so the sameagent.yamlworks on a French or German Windows. - Put each path between single quotes in YAML, so that the backslashes and
the
%are kept as they are. - A malformed path stops the agent at startup with the key and the reason. A
path that Windows does not know (the software is not installed, the name is
misspelt) is reported once in the log,
performance counter not available, and the other counters are still read; it is tried again every ten periods. - Rates (
…/sec) need two readings: they appear from the second period on. - At most 200 counters are read. On Linux, macOS and FreeBSD the setting is accepted and nothing is read.
To see the exact names on the machine, run typeperf -q MDaemon (or any
other object name) in a command prompt.
Hyper-V hosts
Section titled “Hyper-V hosts”Installed on a Hyper-V host — Windows Server with the Hyper-V role, or
Windows with Hyper-V enabled — the agent finds the vmms service and reads
Hyper-V’s own performance counters, with nothing to configure. The agent’s
monitor page then shows a Hyper-V panel: how many virtual machines report
healthy, the host’s real CPU load, the memory each running VM holds, and the
throughput, latency and errors of each virtual disk.
| Counter | Series | Labels |
|---|---|---|
\Hyper-V Virtual Machine Health Summary\Health Ok, …\Health Critical | hyperv_vms_health_ok, hyperv_vms_health_critical: virtual machines in each state | |
\Hyper-V Hypervisor Logical Processor(_Total)\% Total Run Time | hyperv_host_cpu_percent: the host’s load, guests included | |
\Hyper-V Hypervisor Logical Processor(_Total)\% Guest Run Time | hyperv_host_cpu_guest_percent: the share spent running guests | |
\Hyper-V Hypervisor\Partitions | hyperv_partitions: running VMs plus the host itself | |
\Hyper-V Dynamic Memory VM(*)\Physical Memory, …\Guest Visible Physical Memory | hyperv_vm_physical_memory_megabytes, hyperv_vm_guest_visible_memory_megabytes | vm |
\Hyper-V Dynamic Memory VM(*)\Current Pressure | hyperv_vm_memory_pressure_percent: memory demand over memory assigned; above 100 the VM wants more than it has | vm |
\Hyper-V Virtual Storage Device(*)\Read Bytes/sec, …\Write Bytes/sec | hyperv_vhd_read_bytes_per_second, hyperv_vhd_write_bytes_per_second | disk |
\Hyper-V Virtual Storage Device(*)\Latency, …\Queue Length | hyperv_vhd_latency_seconds (average), hyperv_vhd_queue_length | disk |
\Hyper-V Virtual Storage Device(*)\Error Count | hyperv_vhd_errors_total, a counter | disk |
On a Hyper-V host, \Processor(_Total)\% Processor Time only measures the
host’s own partition, not what the virtual machines consume: the hypervisor’s
logical processor counter is the one that tells whether the machine is
saturated. vm is the virtual machine’s name as Hyper-V Manager shows it;
disk is the instance name Windows gives each virtual disk, derived from its
file path. The _Total instances are left out, and at most 200 VMs or disks
are read per counter.
Three built-in rules apply:
- Hyper-V VM health critical: Hyper-V reports a VM in critical health for five minutes (Warning) — most often a disk that ran out of space.
- Hyper-V host CPU high: the logical processors more than 90 % busy for fifteen minutes (Advisory).
- Hyper-V virtual disk errors: a virtual disk reported I/O errors in the last hour (Advisory).
Windows event log alerts
Section titled “Windows event log alerts”The Windows agent can also raise alerts on Windows events (failed logons, unexpected reboots, crashed services, disk errors): rules are set per machine on its device page, none exists by default. See Windows event log alerts.
Run the agent in Docker / on another network
Section titled “Run the agent in Docker / on another network”The agent is also published as a container image, with the same tags as the
server: ghcr.io/laupernoe/dumbmonit-agent:latest (last release), :edge (last
commit on main). It is a FROM scratch image holding the agent binary and
the Mozilla CA bundle, configured entirely by environment variables. The
repository ships a ready-to-use docker-compose.agent.yml:
DUMBMONIT_AGENT_URL=https://monitor.example.org \DUMBMONIT_AGENT_TOKEN=dmon_… \DUMBMONIT_AGENT_HOSTNAME=docker-host-1 \docker compose -f docker-compose.agent.yml up -dWhat the container sees, and what to mount:
| Need | What to do | Why |
|---|---|---|
| CPU, memory, load, disk I/O of the host | Nothing | /proc/stat, /proc/meminfo, /proc/loadavg and /proc/diskstats are host-wide inside any container. |
| Host name | DUMBMONIT_AGENT_HOSTNAME | A container’s hostname is a random id; the monitor would be named after it. |
| Containers of the host | - /var/run/docker.sock:/var/run/docker.sock:ro | Optional. Read-only lists and inspects; Restart/Update actions need it writable and DUMBMONIT_AGENT_COMMANDS=true. Without the socket, the agent simply reports no containers. |
| Filesystem usage | - /:/host:ro (or the mounts you care about) | The agent reports the mounts visible inside the container, under their container path. |
| Host network counters | network_mode: host | Interfaces are namespaced: without it the agent sees one veth. |
| Ping monitors relayed through this agent | sysctls: net.ipv4.ping_group_range: "0 2147483647" | ICMP uses an echo socket, which this sysctl allows: no capability needed. See ICMP ping without a capability. |
| Private CA | - ./ca-bundle.crt:/etc/ssl/certs/ca-certificates.crt:ro | Server behind an internal reverse proxy, monitors with self-signed certificates. |
| Keeping the binding across recreations | - agent-state:/var/lib/dumbmonit-agent and DUMBMONIT_AGENT_SECRET_FILE=/var/lib/dumbmonit-agent/agent-secret | The binding secret lives in the container’s filesystem, which a docker compose up --force-recreate throws away. Without the volume, the agent comes back unbound and someone has to allow re-enrolment from the monitor page each time. |
pid: host and privileged: true are never needed: the agent collects no
per-process metrics.
Relay mode: monitor another site
Section titled “Relay mode: monitor another site”An agent can do more than watch its own machine. With relay: true (or
DUMBMONIT_AGENT_RELAY=true), the server delegates probes to it: monitors
whose Reached through setting names this agent are not polled by the server
but by the agent, from its own network, with exactly the same collectors —
SNMP, Proxmox VE, Proxmox Backup Server, Synology, HTTP, TCP, DNS, ping, TLS.
The results land under the monitor as if the server had polled it: same
metrics, same last probe, same alerts. A remote office, a second homelab or
a customer’s network thus show up in the one DumbMonit you already have,
without a VPN and with nothing to open on the remote side.
How to set it up, step by step, is in Monitor a remote site.
Binding: one machine, one agent
Section titled “Binding: one machine, one agent”An enrollment token says this machine is allowed to talk. It does not say
which machine is talking — that is what the identity key
(/etc/machine-id, or the host name) is for, and that key is no secret: it can
be read on any machine in seconds.
So on the first batch the server hands each machine a binding secret of its
own. It keeps only a fingerprint; the agent writes the secret to
/etc/dumbmonit/agent-secret with mode 0600 and presents it in the
X-DumbMonit-Agent-Secret header on every request afterwards — measurements,
container commands and relayed probes alike. From then on:
- no other machine can push measurements under this monitor, or rewrite its host name and operating system;
- no other machine can fetch — and therefore consume — the Restart and Update commands queued for it.
The monitor page shows where a machine stands, under Agent → Binding:
| State | What it means | What to do |
|---|---|---|
| Bound | The agent holds a secret of its own. | Nothing. |
| Not bound — re-enrol this host | A host enrolled before binding existed, and never bound since. The server refuses its measurements, container commands and relayed probes. | Click Allow re-enrolment on the monitor page, then restart the agent. If the agent predates binding, re-run the install command on the host first. |
Hosts that predate binding
Section titled “Hosts that predate binding”To bring such a host back:
- If its agent predates binding, re-run the install command on it (the agent binary must be able to store a binding secret).
- Open its monitor page and click Agent → Allow re-enrolment. This opens a one-hour window.
- Restart the agent, or wait for its next retry: that batch binds the host with a fresh secret and closes the window. A token other than the one that enrolled the host must still be able to enrol (not used up, not expired).
Until then, the agent’s log carries the server’s explanation:
WARN this agent is not recognised for this machine reason: This machine was enrolled before agent binding and was never bound to its agent installation, so the server no longer accepts it. Re-enrol this host: open its device page in DumbMonit, click 'Allow re-enrolment', then restart the agent (re-run the install command first if the agent is older than binding).Re-enrolment after a reinstall
Section titled “Re-enrolment after a reinstall”If the machine is rebuilt, its disk replaced, or its agent container recreated without the volume holding the secret, the agent comes back without it and the server refuses its batches. That refusal is deliberate: a machine that is bound stays bound until a human says otherwise, otherwise the binding would protect nothing.
The way back in is on the monitor page: Agent → Allow re-enrolment. It opens a one-hour window during which an agent presenting a valid enrollment token binds that machine again, with a fresh secret; the old secret stops working, and the window closes as soon as it is used. The agent retries on its own in the meantime, so there is nothing to restart on the machine — and nothing is lost, since its readings stay buffered until the link is accepted again.
Until someone opens that window, the agent’s log says what to do:
WARN this agent is not recognised for this machine reason: This machine is already enrolled and bound to another agent installation. If you reinstalled it, open its device page in DumbMonit and click 'Allow re-enrolment', then restart the agent.The agent forgets its secret only on that exact refusal. Any other 403, for
instance from a reverse proxy or a web application firewall in front of
DumbMonit, is logged and retried with the secret kept.
Enrolment tokens
Section titled “Enrolment tokens”- Tokens are created when adding a monitor of type agent (single use, or reusable for several machines) and by the Active Directory kit. The server keeps only a fingerprint: the clear token is shown once, in the creation response, with the two install commands.
- The agent sends it as
Authorization: Bearer dmon_…onPOST /api/ingest, over the encrypted agent channel. It never appears in the agent’s logs, not even truncated. - Settings → Agents lists tokens by their opaque identifier (
tok_…, never the secret), scope, creation date and last use. Revoke stops every agent that uses that token at its next push; the machines then become unreachable after three missed periods. Re-run the installer with a new token to re-enrol them. - Machines are separate monitors: revoking a token does not delete them.
Single use or fleet
Section titled “Single use or fleet”A token’s scope is chosen when it is created, and it is a choice, not a default that happens to you:
| Scope | What it enrols | When |
|---|---|---|
| Single use (the default) | Exactly one machine. | One install. An install command lives on in shell history, in a ticket, in a chat log; a single-use one is worth nothing once used. |
| Reusable for a fleet | As many machines as you allow — a number, or no limit. | A playbook, a machine image, a batch of installs. |
A new machine enrolling with a fleet token can wait for an administrator’s approval, and a fleet token that enrolled nothing for 30 days goes to sleep: see Protecting enrolment.
Both can carry a deadline in days. A deadline only stops enrolments: machines already enrolled keep reporting after it passes, which is what you want for an install window. Revoking is what cuts a fleet off.
A token that can no longer enrol — used up, or past its deadline — is shown as Enrols no more in Settings → Agents. Its machines keep reporting; it just cannot let a new one in.
Outages and diagnostics
Section titled “Outages and diagnostics”When the server is unreachable, the agent keeps measuring and buffers its
readings in memory with their original timestamps. They are sent as soon as
the link is back: a server restart leaves no hole in the graphs. The buffer is
bounded (max_buffered_samples, about an hour by default) and drops the oldest
readings when full. Retries back off from 2 to 30 seconds with jitter, and resume at once when
the relay channel reaches the server again. Measurements still waiting at
shutdown are kept on disk (pending-samples.json, next to the binding secret)
and sent at the next start.
dumbmonit-agent --dry-run # print the readings, send nothingdumbmonit-agent --once # send one batch then exitjournalctl -u dumbmonit-agent -f # service log (systemd)tail -f /var/log/dumbmonit-agent.log # service log (OpenRC)Common errors
Section titled “Common errors”| Symptom | Likely cause |
|---|---|
| Installer says “could not reach the server” | Wrong URL or token. The configuration is written: fix agent.yaml, then restart the service. |
| Download fails with 404 | The server has no agent binaries (DUMBMONIT_AGENT_DIR empty). Use --bin=PATH with a binary you built. |
| Installer stops with “checksum mismatch” | The command was copied before a server upgrade (copy a fresh one when adding a server with the agent), or the binary received is not the one the server serves: a proxy or a cache in the way, or a tampered download. |
| Installer stops with “no expected checksum” | The command has no --sha256 for this platform: an old command, or a macOS binary (not shipped by the server). Copy a fresh command, or pass --sha256=<platform>-<arch>:<hex> from the release’s .sha256. |
| Machine never appears | The push goes to /api/ingest on the URL in the install command; behind a reverse proxy, make sure it is forwarded and that bodies up to 16 MB are allowed. |
| Unreachable although the agent runs | The token was revoked, or the pushes are rejected. Check journalctl -u dumbmonit-agent: the error is logged there. |
| Log says “this agent is not recognised for this machine” | The machine is bound to an agent installation whose secret this one does not have — a reinstall, or a container recreated without its state volume. Open the monitor page and click Allow re-enrolment; see Re-enrolment after a reinstall. |
| Log says “this machine was enrolled before agent binding” | A host that never bound to its agent. Click Allow re-enrolment on its monitor page, then restart the agent; see Hosts that predate binding. |
| Log says “this agent is too old to be bound to its machine” | The agent binary cannot store a binding secret. Re-run the install command on the host; if the monitor already exists, click Allow re-enrolment first. |
| Log says “this enrollment token has already enrolled all the machines it was allowed to” | A single-use token being reused. Create a new one, or tick Several machines with this command when adding the server. |
| Windows | The Windows service parts had not been exercised in the project’s build image at the time of writing; report issues on GitHub. |
Docker containers
Section titled “Docker containers”When the agent can read the Docker socket (docker: true, the default, and a
user in the docker group or root — usermod -aG docker dumbmonit, then
restart the agent), every container becomes a handful of series, labelled
container and image (the name:tag shown by docker ps). If the monitor
page says “No Docker on this machine” while Docker is installed, that is the
first thing to check.
| Metric | Meaning |
|---|---|
container_up | 1 running, 0 otherwise |
container_health | 0 no healthcheck, 1 healthy, 2 unhealthy, 3 starting |
container_restart_count | Docker’s restart counter for the container |
container_started_seconds | Seconds since the container last started (0 when stopped) |
container_image_age_seconds | Seconds since the image was built |
container_update_available | 1 when the registry has a newer image for the same tag, 0 when up to date, -1 unknown |
The update check compares the digest of the local image with the one the
registry announces for the tag (HEAD /v2/<name>/manifests/<tag>, anonymous
token flow on Docker Hub and GHCR). It runs in the background at most once an
hour per image and never delays the 30-second cycle; a private registry that
refuses anonymous access simply reads as -1. Disable it with
docker_update_check: false (DUMBMONIT_AGENT_DOCKER_UPDATE_CHECK).
Built-in rules: Container stopped (container_up == 0 for 2 minutes),
Container unhealthy (container_health == 2 for 3 minutes), Container
restarting (three or more restarts in 15 minutes), Container update
available (info, after one hour, repeated daily). The alert names the
container.
Actions
Section titled “Actions”On the monitor page, each container has two actions and two switches. Actions
travel through a command channel: the server queues the command, the agent
fetches it after its next batch (GET /api/agent/commands, same enrollment
token), runs it, and reports the outcome (queued → running → done | failed),
with a short log you can unfold in the interface. Commands older than ten
minutes are never executed; the agent runs one command at a time.
A queued command can be cancelled from the interface as long as the agent has not picked it up. A command nobody came to fetch expires after ten minutes, on the server, whether or not the agent ever polls — so a machine whose agent is stopped never keeps a stale “Queued” forever, and never blocks the next action on that container.
The agent tells the server, with every batch, whether it accepts commands.
When it does not — an agent older than the command channel, or installed with
commands: false — the monitor page says so instead of showing Restart and
Update, the API refuses the commands (409), and policies do not queue
anything. Reinstall the agent with the current installer to enable actions.
- Restart —
docker restartfor a running container,docker startfor a stopped one, then a 20-second check that it runs. - Update now — pulls the tag the container already uses, creates a new
container with the same configuration (environment, mounts, ports, labels,
networks, restart policy), stops the old one, swaps the names, starts the new
one and waits for its healthcheck (or 20 seconds plus a running check when
the image has none). If anything fails, the agent rolls back: the new
container is removed and the old one restarted, and the command reports
failedwith the reason. On success the old container is removed and, when “prune” is on, the old image too — unless another container still uses it.
Compose-managed containers (labels com.docker.compose.*) are updated all the
same, but the result warns that the next docker compose up will recreate the
container from the compose file. Containers whose name starts with dumbmonit
or dumbmonit are refused: the monitor never updates itself.
Policies
Section titled “Policies”A per-container policy is evaluated by the server every minute:
- Restart if down — a container reporting
container_up == 0is restarted automatically, at most once every ten minutes. - Auto-update in maintenance windows — a container with an update available is updated automatically, but only while a maintenance window (silence) covers the monitor, at most once every six hours. Schedule one on the Alerts page, or silence the monitor from its page. Policies default to pruning the old image and to maintenance windows only.
Two places set them:
- the Docker strip under the monitor header: “28 containers · 27 running · 3 updates available · policies: 2 auto-restart, 0 auto-update”, the last action and its status, Manage containers (opens the list below) and Policies…, a table of every container with the two switches and an “Apply to all” row — the quickest way to turn auto-restart on for a whole machine;
- the switches inside each row of the Containers section, next to the Restart and Update now buttons.
Every switch saves on its own (PUT /api/targets/<id>/containers/<name>/policy);
a failure only concerns that row and is shown next to it.
Automatic actions show in “Recent actions” as requested by policy.
Safety rules
Section titled “Safety rules”- The agent only acts when
commands: true(the default); set it tofalsefor a machine that must only be measured. - The agent fetches commands itself, with its enrollment token — nothing connects to the machine. They expire after ten minutes and run one at a time; the agent never pulls a different tag than the one the container already uses.
- An update that does not come back healthy is rolled back to the previous container, and the old image is only removed once the new container runs.
- Containers named
dumbmonit*orezymonit*are refused, by hand or by policy: the monitor never acts on its own containers. The check is made twice — the server refuses to queue the command, and the agent refuses to run it — and the agent makes it again on the name Docker reports after inspecting the container, so naming one by its id changes nothing. The agent also refuses to act on its own container, whatever it is called: it would not be there to finish the job or report on it. - Compose-managed containers are updated, but the next
docker compose uprecreates them from the compose file: bump the tag there too.
Plakar backups
Section titled “Plakar backups”The agent can watch Plakar klosets (repositories): every
ten minutes, in the background, it runs plakar at <kloset> ls and info on
each one and re-emits the last reading each cycle.
Plakar is detected, never assumed. At startup and then on every cycle
until it finds something, the agent looks for the plakar binary (on PATH,
or plakar_bin) and for klosets in the standard locations. When there is no
binary, no kloset and no trace of Plakar (~/.cache/plakar), the agent emits
no backup_* series at all, says so once at debug level, and the monitor
page shows no Backups panel — nothing to configure on machines without Plakar.
To skip the detection entirely:
plakar: false # or DUMBMONIT_AGENT_PLAKAR=falseDiscovery
Section titled “Discovery”Nothing to configure: when plakar_klosets is empty, the agent finds the
klosets itself, in this order, and reads each location only once:
- the agent’s own configuration directory —
$XDG_CONFIG_HOME/plakar, or~/.config/plakarof the account it runs under (plakar_homewhen set); /root/.config/plakar;/home/*/.config/plakar, alphabetically;/var/lib/plakar, when that directory exists.
In each directory it reads stores.yml (or stores.yaml; the legacy
plakar.yml is accepted too) and takes every entry of the stores map (also
repositories and klosets) that has a location. Only the location is read:
passphrases and rclone secrets that sit next to it are never kept nor logged.
Each entry becomes the kloset @<name>, read with plakar -configdir <dir> at @<name>, so a kloset created by a user is read even when the agent runs as
root. ~/.plakar of every account is added when it exists (the default kloset
of plakar create). Two accounts naming different klosets the same way are told
apart as @name and <user>:@name.
The result is logged once at info (and again only when it changes). Two
gauges, emitted only once Plakar is detected, let the interface tell the
remaining cases apart: backup_plakar_present (1 when the binary runs) and
backup_klosets_found (number of klosets read). The monitor page then says
“Plakar is installed but no kloset was found”, or lists the klosets. When
klosets exist but the binary does not run (a service PATH without plakar,
typically), they are listed as unreadable with a hint to set plakar_bin, and
the agent warns once.
To watch a fixed list instead, name it — discovery is then skipped:
plakar: true # default: detect Plakar; false hides it entirelyplakar_bin: /usr/bin/plakar # default: "plakar" on PATHplakar_klosets: # explicit list: overrides discovery - /srv/backups/main - "@diskext" # a named store of the agent's accountplakar_home: /home/noe # optional: HOME (and config dir) used when running plakarplakar_interval_secs: 600 # minimum 60| Variable | Effect |
|---|---|
DUMBMONIT_AGENT_PLAKAR | true / false — watch Plakar backups when detected (default true) |
DUMBMONIT_AGENT_PLAKAR_BIN | Path of the plakar binary |
DUMBMONIT_AGENT_PLAKAR_KLOSETS | Klosets, comma-separated (overrides discovery) |
DUMBMONIT_AGENT_PLAKAR_HOME | Home directory used when running plakar |
DUMBMONIT_AGENT_PLAKAR_INTERVAL_SECS | Seconds between two readings |
DUMBMONIT_AGENT_DOCKER_UPDATE_CHECK | true / false — registry check for container updates |
DUMBMONIT_AGENT_COMMANDS | true / false — accept actions from the server |
An encrypted kloset needs its passphrase: put it in the store entry
(plakar store set <name> passphrase=…, which is what plakar store add does)
or export PLAKAR_PASSPHRASE in the agent’s service environment. Without it
the kloset reads as “Cannot be opened” (backup_last_status = 0), the agent
logs why once (and again when it recovers), and the rule “Plakar kloset
unreadable” warns after 30 minutes. A kloset you do not care about (an
offsite copy only reachable by hand, say) is left out with plakar_klosets.
The agent runs plakar as root, with HOME set to the account where the
kloset was found, -datadir pointing to that account’s
~/.local/share/plakar (its plugins, rclone included) and -cachedir in the
agent’s own cache directory (/var/cache/dumbmonit-agent under systemd).
Installs made before October 2026 hid /home from the agent
(ProtectHome=yes): discovery then found nothing, and the agent now warns
about it. Run the install command again to get the current unit
(ProtectHome=read-only).
Metrics and rules
Section titled “Metrics and rules”Metrics, labelled kloset and source (the directory that was backed up):
backup_last_success_seconds (age of the newest snapshot),
backup_snapshot_count, backup_last_status (1 when the newest snapshot has
no error, 0 when it has errors or the kloset cannot be opened),
backup_size_bytes{kloset} (storage size of the repository), plus
backup_plakar_present and backup_klosets_found described above. None of
them exists while Plakar is not detected, so the built-in rules below cannot
fire on a machine without Plakar.
Built-in rules: Plakar backup too old (advisory, no snapshot for more than two days) and Plakar backup failed (warning). The monitor page shows one block per kloset — age of the last backup, snapshot count, size, last status — and one line per source, with a “Backup too old” hint past two days.
Run a first backup
Section titled “Run a first backup”With Plakar 1.1 installed on the machine (plakar version), as the user who
will own the backups:
# 1. a kloset store, named so the agent finds it in ~/.config/plakar/stores.ymlplakar store add diskext /mnt/diskext/plakar passphrase=… # or any location Plakar acceptsplakar at @diskext create # add -plaintext to skip encryption
# 2. a first backup, then check itplakar at @diskext backup /srv/photosplakar at @diskext lsplakar at <path> create and plakar at <path> backup <dir> work the same with
a plain directory instead of @name; plakar create without at uses
~/.plakar, which the agent also finds. Schedule plakar at @diskext backup /srv/photos with cron or a systemd timer: the agent reports the age of the
newest snapshot, and alerts when a day goes by without one — two days for the
built-in rule.
To try it locally with an unencrypted kloset:
mkdir -p /tmp/plakar-demo-src && echo hello > /tmp/plakar-demo-src/a.txtplakar at /tmp/plakar-demo create -plaintextplakar at /tmp/plakar-demo backup /tmp/plakar-demo-srcplakar at /tmp/plakar-demo lsthen add plakar_klosets: ["/tmp/plakar-demo"] to agent.yaml (a bare path
outside a home is not discovered) and check with dumbmonit-agent --dry-run
that backup_* samples appear.
restic and Borg backups
Section titled “restic and Borg backups”The agent can watch restic and
Borg repositories: every ten minutes, in the
background, it opens each repository you declare and reports whether it could
be read and how old its newest snapshot is. Nothing is discovered: a
repository has no standard location and cannot be opened without its password,
so each one is listed in agent.yaml, with the way to get its password.
The password never leaves the machine. The agent hands it to restic or
borg through the environment of the child process only; it is never logged,
never sent to the server, and the series only carry the repository’s name.
restic_repos: - name: nas # the "repo" label; default: the repository, without credentials repository: sftp:backup@nas.lan:/srv/restic # passed as RESTIC_REPOSITORY, never on the command line password_file: /etc/dumbmonit/restic-nas.pass host: web-01 # optional: only this host's snapshots (--host), for a shared repository args: ["-o", "sftp.command=ssh -i /etc/dumbmonit/id_backup -o UserKnownHostsFile=/etc/dumbmonit/known_hosts backup@nas.lan -s sftp"] - name: b2 repository: s3:https://s3.eu-central-003.backblazeb2.com/my-bucket password_command: cat /etc/dumbmonit/restic-b2.pass # or any command that prints it (RESTIC_PASSWORD_COMMAND) env: # extra environment for this repository only AWS_ACCESS_KEY_ID: "…" AWS_SECRET_ACCESS_KEY: "…"borg_repos: - name: offsite repository: ssh://borg@offsite.example.net/./backups # passed as BORG_REPO password_file: /etc/dumbmonit/borg-offsite.pass # read by the agent, passed as BORG_PASSPHRASE glob_archives: "web-01-*" # optional: only these archives (--glob-archives) env: BORG_RSH: "ssh -i /etc/dumbmonit/id_borg -o UserKnownHostsFile=/etc/dumbmonit/known_hosts"restic_bin: restic # default: on PATHborg_bin: borgbackup_repos_interval_secs: 600 # minimum 60Keep agent.yaml and the password files readable by root only
(chmod 600), as the installer does for agent.yaml. password_command
maps to RESTIC_PASSWORD_COMMAND or BORG_PASSCOMMAND. Anything the tools
read from their environment can go under env (RESTIC_PASSWORD,
BORG_PASSPHRASE, cloud keys, BORG_RSH…), and the agent’s own service
environment is passed through as well.
What the agent runs, read-only in the strictest sense:
restic snapshots --json --latest 1 --no-lock --no-cache: no lock is written, so a read-only or append-only repository can be watched, and apruneholding the exclusive lock does not make the repository look down; no cache either, since the installed service cannot write to/root.borg --bypass-lock list --json --last 1: without--bypass-lock, the reading would fail for the whole duration of everyborg create, which holds the exclusive lock. Borg writes a small security record under~/.config/borg; when that home is not writable (the installed service runs withProtectHome=yes), the agent pointsBORG_BASE_DIRto its private/tmpinstead, unless you setBORG_BASE_DIRyourself. A repository inkeyfilemode then needsBORG_KEY_FILEin itsenv.
Because the service runs with ProtectHome=yes, keep what the tools need
outside the home directories: password files, SSH keys and known_hosts under
/etc/dumbmonit/, as in the example, and local repositories under /srv,
/mnt or /var rather than /home. The readings run one after the other,
inside the memory limit of the service (128 MiB with the agent itself).
Metrics, labelled tool (restic or borg) and repo:
agent_backup_repo_reachable (1 when the last reading succeeded, 0 otherwise:
repository unreachable, password refused, binary missing — the agent logs the
reason once, and again when it changes), agent_backup_repo_last_snapshot_age_seconds
(age of the newest snapshot, absent while the repository is empty), and
agent_backup_repo_last_snapshot_bytes (bytes the newest restic snapshot
processed, restic 0.17 and later). With --latest 1, restic returns the
newest snapshot of each host and path set; the newest of them all is kept, so
set host on a repository shared by several machines.
Built-in rules: restic/Borg backup too old (no snapshot for more than 48 hours) and restic/Borg repository unreachable (for 30 minutes). The monitor page shows a Backup repositories panel, one line per repository.
To check a declaration, run the agent once in the foreground and look for the
agent_backup_repo_* samples:
sudo dumbmonit-agent --config /etc/dumbmonit/agent.yaml --dry-runWireGuard tunnels
Section titled “WireGuard tunnels”On a machine that runs WireGuard, the agent reads every interface and every
peer with wg show all dump, at each cycle. Nothing to configure: without the
wg command (wireguard-tools) or without an interface, no series is emitted
and no panel shown. wg needs root (CAP_NET_ADMIN); the installed service
has it, an agent started by hand under your own account logs the refusal once
and reports nothing.
The dump also holds the interface’s private key and each peer’s preshared key: the agent skips both fields without ever copying them.
Per peer, labelled interface, peer (its public key) and allowed_ips:
agent_wireguard_peer_handshake_age_seconds (seconds since the latest
handshake), agent_wireguard_peer_has_handshake (0 while the peer has never
completed one), agent_wireguard_peer_keepalive_seconds (its persistent
keepalive, 0 when off), and the counters agent_wireguard_peer_rx_bytes and
agent_wireguard_peer_tx_bytes; per interface,
agent_wireguard_interface_peers. For a peer that has never completed a
handshake, the age counts from the moment the agent first saw it: a tunnel
that never came up is silent too, and is reported as such.
WireGuard has no peer names. Give yours in agent.yaml, by public key, and
they become the name label:
wireguard: true # default; false stops even lookingwireguard_bin: wg # default: on PATHwireguard_peer_names: "zonmO992nrIL4FgJ3B/wccmu/SBkoX2kDhSYr8KpgRU=": office-gateway "UsKpqhrXNlYy/+rh2DLAOOEqesn8L8okEwNNorZ09lQ=": laptopThe built-in rule WireGuard peer silent fires when a peer with a
persistent keepalive has not completed a handshake for more than 15
minutes. While a tunnel is up, WireGuard renews its handshake every two
minutes, and a keepalive keeps it up even without traffic; a peer without
keepalive (a phone, a laptop) may idle for days with nothing broken, so it is
left out. Set PersistentKeepalive = 25 on the tunnels that must stay up
(site-to-site links, remote gateways). N is the rule’s threshold: change it in
Alerts → Rules, or remove the and on (…) part of its query to watch
every peer. The monitor page shows a WireGuard panel with each peer, the time
since its last handshake, its traffic and its keepalive.
IPsec tunnels
Section titled “IPsec tunnels”On a machine running strongSwan, the agent reads every configured tunnel with
two read-only commands: swanctl --list-conns, for the CHILD_SAs declared in
swanctl.conf (or ipsec.conf through swanctl --load-all, strongSwan’s own
compatibility layer — the agent does not care which wrote the configuration),
and swanctl --list-sas, for the ones actually installed. Nothing to
configure: without the swanctl command or without a configured connection, no
series is emitted and no panel shown. The agent never sends --initiate,
--terminate or any other command that would change a tunnel’s state; it only
reads what swanctl already shows on the command line.
Reading both commands, rather than only the active one, is what lets a tunnel
that has never come up be reported as down instead of invisible: a CHILD_SA
declared in the configuration but absent from --list-sas counts as down from
the first cycle.
Per tunnel, labelled connection (the IKE_SA’s name) and tunnel (the
CHILD_SA’s name): agent_ipsec_tunnel_up (1 installed, 0 otherwise), and, only
while up, the counters agent_ipsec_tunnel_bytes_in / agent_ipsec_tunnel_bytes_out
and agent_ipsec_tunnel_rekey_seconds (time left before the next rekey).
ipsec: true # default; false stops even lookingipsec_bin: swanctl # default: on PATHThe built-in rule IPsec tunnel down fires when a configured tunnel has no installed CHILD_SA for ten minutes. The monitor page shows an IPsec panel with each tunnel, up or down, its traffic and the time to its next rekey.
Internet speed test
Section titled “Internet speed test”Off by default: unlike everything else the agent measures, this test
consumes bandwidth — and, on a metered or capped connection, money — every
time it runs. Turning it on needs both speedtest: true and a download
address; without the address, nothing runs even if speedtest is true.
No server is chosen for you. DumbMonit does not ship a default endpoint and never contacts a third party unless you name one: you decide whose server your line is measured against, and who sees the agent’s address while it runs.
The test is plain HTTP, read-only on the far end: a GET on
speedtest_download_url, read chunk by chunk and stopped as soon as
speedtest_download_bytes_cap bytes have been read (default 50 MB,
whichever ends first), gives the download rate; a POST of
speedtest_upload_bytes bytes (default 10 MB) to speedtest_upload_url, if
set, gives the upload rate; a handful of HEAD requests on the download
address give the latency (the smallest round trip) and the jitter (the spread
between the smallest and the largest). A server that refuses HEAD simply
leaves latency and jitter unmeasured — the rest of the test does not depend on
it.
This works against a self-hosted LibreSpeed
backend (backend/garbage.php?ckSize=N for download, backend/empty.php for
upload), against Cloudflare’s speed test endpoints
(https://speed.cloudflare.com/__down?bytes=N and
https://speed.cloudflare.com/__up, which anyone may call without an
account), or against any plain HTTP file and upload endpoint of your own —
there is no vendor-specific protocol to implement, and no new dependency
beyond the HTTP client the agent already uses.
speedtest: false # default: opt-in, this test uses bandwidthspeedtest_download_url: https://speed.cloudflare.com/__down?bytes=50000000speedtest_upload_url: https://speed.cloudflare.com/__upspeedtest_download_bytes_cap: 50000000 # default; read at most this many bytesspeedtest_upload_bytes: 10000000 # default; sent in one blockspeedtest_interval_secs: 21600 # default: 6 hours, minimum 300speedtest_min_download_mbps: 50 # optional: arms "Internet download rate low"speedtest_min_upload_mbps: 10 # optional: arms "Internet upload rate low"speedtest_max_latency_ms: 80 # optional: arms "Internet latency high"The test runs in the background, at most once per speedtest_interval_secs
(default 6 hours, never less than 300 seconds: a smaller value stops the agent
at start-up); one test also runs a few seconds after each agent start;
the collection cycle only reads its last result, exactly like the restic/Borg
and Plakar readings. agent_speedtest_ok is 0 when the download itself failed
(unreachable address, timeout, HTTP error) — the only failure that stops the
test from measuring anything at all.
The three threshold options are deliberately optional: without
speedtest_min_download_mbps, no one at DumbMonit knows what “slow” means for
your line, so Internet download rate low, Internet upload rate low and
Internet latency high stay silent until you set the one that applies.
Each threshold is published as its own series
(agent_speedtest_download_min_bytes_per_second, and so on), which is what
the built-in rules compare the measurement against; change the threshold in
agent.yaml, not in Alerts → Rules. The monitor page shows a speed test
panel with the last download and upload rate, latency, jitter and how long ago
the test ran.
TLS certificates (Certbot / Let’s Encrypt)
Section titled “TLS certificates (Certbot / Let’s Encrypt)”The agent does not read /etc/letsencrypt directly. Every certificate
Certbot renews is served somewhere — a web server, a mail server, a reverse
proxy — and that is where DumbMonit reads it: add a
TLS certificate monitor (or a
Website or web API one, for a site already watched that
way) pointed at the port the certificate is actually presented on. It reads
the live certificate, not a file on disk, so it also catches a renewal that
succeeded in /etc/letsencrypt but was never picked up by the service (a
reverse proxy that needs a reload to see the new file is the usual case, and
exactly the gap that matters).
Reading /etc/letsencrypt/live/*/cert.pem, the renewal configuration and the
Certbot log from the agent was considered and set aside: it would duplicate
the expiry date the TLS monitor already reports, while adding a second,
host-local path to the same information — one more thing to keep in sync
after a certificate is moved or a domain renamed. A host where Certbot runs
but serves nothing DumbMonit can reach directly (an internal CA step, a DNS
validation hook) is the one case this leaves uncovered; it is also the case
where a TLS monitor could not help either, since nothing on that host speaks
TLS to probe.