Skip to content

Metrics

DumbMonit stores every measurement in VictoriaMetrics and reads it back with MetricsQL, through the server’s metrics proxy. This page describes the naming, the labels and the retention. Per-type metric lists are on the monitor pages.

Every metric is prefixed dumbmonit_, so DumbMonit can share a VictoriaMetrics instance with other tools without collisions. Names follow dumbmonit_<what>_<unit>:

PatternExamples
…_bytes, …_bytes_total, …_bytes_useddumbmonit_memory_bytes_total, dumbmonit_filesystem_used_bytes
…_percentdumbmonit_cpu_usage_percent, dumbmonit_pbs_datastore_used_percent
…_seconds, …_age_secondsdumbmonit_probe_duration_seconds, dumbmonit_proxmox_backup_last_age_seconds
…_celsius, …_volts, …_hertz, …_wattsUPS and Synology readings
…_infoPresence series (value 1) whose information is in labels: dumbmonit_system_info, dumbmonit_pbs_version_info
…_status, …_up, …_runningEnumerations or booleans: dumbmonit_if_oper_status, dumbmonit_proxmox_guest_running

Source-specific families carry their source as a second prefix: dumbmonit_proxmox_*, dumbmonit_pbs_*, dumbmonit_synology_*, dumbmonit_probe_* (services). SNMP profiles and the agent use plain names (dumbmonit_if_octets_in, dumbmonit_cpu_usage_percent).

Counters (interface octets and packets, disk bytes, pages printed) are stored raw, as the monitor reports them: apply rate() in your own queries to get a throughput. The monitor page charts every dumbmonit_* series of the monitor as stored, so a counter appears as a rising line. A reboot shows as a counter reset, never as a fake spike.

After every successful probe, the server writes dumbmonit_up = 1 for the monitor. Nothing is written when a probe fails: the series simply stops, and the “Monitor unreachable” rule measures the age of the last sample (time() - tlast_over_time(dumbmonit_up[7d])), keeping only the freshest series of each monitor (bottomk by (target) (1, …)): a series no longer written after a tag or label change stays in the 7-day window and must not count.

For services (http, tcp, dns, ping, tls), dumbmonit_up = 1 means “the monitor ran”, and dumbmonit_probe_success says whether the service is fine. The two signals are kept apart on purpose: a probe_success of 0 says the service is down, an interruption of up says the monitoring is down.

When a monitor is probed from several locations, dumbmonit_probe_success carries the quorum decision, and these series add the detail:

MetricMeaning
dumbmonit_probe_location_success{location}1 or 0 for each location (server or agent-<id>)
dumbmonit_probe_location_duration_seconds{location}Response time seen from each location
dumbmonit_probe_degraded1 while only a minority of the locations fail (no down alert)
dumbmonit_probe_locations_failingLocations failing in the last decision
dumbmonit_probe_locations_countedLocations counted in the last decision (silent ones are left out)

Every 15 minutes the server computes the security score of each monitor whose kind has security checks, and writes dumbmonit_security_score (0–100, absent while nothing can be rated) and dumbmonit_security_checks_failed, both labelled target.

LabelSet byMeaning
targetserverThe monitor id, as in /api/targets/{id}. Every rule and every chart selects on it.
hostserverThe monitor name.
tag_<key>serverThe free tags of the monitor. Type options (insecure_tls, port…), tags whose key looks like a secret (matched per word: key, pass, cert, private, pem…) and values over 256 characters are not copied. Never put a secret in a tag.
probeserviceshttp, tcp, dns, ping or tls.
per-series labelscollectorifname, mountpoint, device, fstype, core, service, container, node, vmid, name, storage, datastore, group, volume, disk, pool, repo, changes, reason, version, issuer… as listed on each monitor page.

Identity labels always win over collector labels, and for agents they are set on reception: a machine cannot write into another machine’s series.

The embedded VictoriaMetrics keeps 12 months of data (DUMBMONIT_VM_RETENTION) under /data/vm, in the dumbmonit-data volume. The server batches its writes and flushes them every 5 seconds (DUMBMONIT_FLUSH_INTERVAL_SECS). Deleting a monitor does not delete its series; they age out.

Raw retention is also set in Settings → System → Data & retention. Beyond it, the server keeps 5-minute aggregates for 730 days by default in a second VictoriaMetrics (/data/vm-long): the average under the original metric name, the minimum and maximum as <name>:5m_min and <name>:5m_max, and the sum and count of availability metrics as <name>:5m_sum and <name>:5m_count (counters keep their last value of each window). A query whose range goes past the raw retention is answered from them automatically, at 5-minute resolution. See Sizing and the configuration reference.

Alert history is kept 90 days and the baselines of series that disappeared for 60 days, both in SQLite (see the configuration reference).

  • Through the server: GET /api/metrics/query and GET /api/metrics/query_range, with the session cookie. Range results are capped at 2,000 points per series.
  • From an existing Prometheus or Grafana: GET /federate, GET /metrics and the /prometheus data source, with an API token — below.
  • Directly, in development: the overlay docker-compose.dev.yml publishes the embedded VictoriaMetrics on http://localhost:8428, with its own UI at http://localhost:8428/vmui/. In production it listens on the container’s loopback only; set DUMBMONIT_VM_LISTEN=0.0.0.0:8428 and publish the port yourself if you want Grafana or another tool on the same data.
# Interface throughput in bits per second
rate(dumbmonit_if_octets_in{target="4"}[5m]) * 8
# Availability of every service over 30 days
avg by (host) (avg_over_time(dumbmonit_probe_success[30d]))
# Every device that has not reported for 5 minutes
bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d])) > 300

If you already run a Prometheus or a Grafana, you do not need a second stack. Three routes sit outside /api, where a scraper looks for them:

RouteWhat it returns
GET /metricsThe health of the DumbMonit instance itself: scheduler, probes, writes, alerting, notifications, agents, database. A few dozen series, always the same number — never one per monitor.
GET /federateThe measurements, selected by match[], exactly as Prometheus federates another Prometheus: the latest point of each selected series.
GET /prometheus/api/v1/…The read half of the Prometheus API, relayed to the store: what a Grafana data source calls.

All three take Authorization: Bearer dmt_… with the read scope — a token created in Settings → API & assistants, the same kind the MCP server uses. No cookie is involved, so no X-Requested-With header is needed; a session cookie is accepted too, so you can open any of them from a signed-in browser. A token is limited to 120 calls a minute: plenty for a scrape every 15 seconds, and enough for a Grafana dashboard of a few dozen panels — give a busy dashboard its own token rather than sharing one with a scraper.

# prometheus.yml — DumbMonit's own health, then the measurements it collects.
scrape_configs:
- job_name: dumbmonit
metrics_path: /metrics
scheme: http
authorization:
type: Bearer
credentials: dmt_0123456789abcdef0123456789abcdef
static_configs:
- targets: ['dumbmonit.lan:8080']
- job_name: dumbmonit-federate
metrics_path: /federate
scheme: http
honor_labels: true
scrape_interval: 60s
authorization:
type: Bearer
credentials: dmt_0123456789abcdef0123456789abcdef
params:
'match[]':
- '{__name__=~"dumbmonit_.*"}'
static_configs:
- targets: ['dumbmonit.lan:8080']

honor_labels: true keeps the target and host labels DumbMonit sets instead of overwriting them with the scrape job’s own. Put the token in a file and use authorization: {type: Bearer, credentials_file: /etc/prometheus/dumbmonit.token} if you would rather not have it in the configuration.

Without match[], /federate selects {__name__=~"dumbmonit_.*"} — every series DumbMonit writes, and nothing belonging to another tool that shares the same VictoriaMetrics. Narrow it to keep a federation small: {__name__=~"dumbmonit_.*",target="4"} for one monitor, {__name__="dumbmonit_up"} for availability only. At most 10 selectors per request, and a response over 8 MiB is refused with a message telling you to narrow it — split a large fleet into several jobs with narrower selectors rather than one that asks for everything. max_lookback=15m widens the window in which the last point is looked for, for monitors probed less often than every five minutes.

Terminal window
curl -H 'Authorization: Bearer dmt_…' https://localhost:8080/metrics
curl -H 'Authorization: Bearer dmt_…' -G https://localhost:8080/federate \
--data-urlencode 'match[]={__name__=~"dumbmonit_.*",target="4"}'

Point Grafana at DumbMonit, not at the VictoriaMetrics it runs: the child process listens on the container’s loopback, has no authentication of its own, and exposes the routes that delete series as readily as the ones that read them. DumbMonit relays the read half of the Prometheus API under /prometheus, behind the same token, so Grafana talks to a normal Prometheus data source:

  1. Connections → Data sources → Add → Prometheus.
  2. URL: https://dumbmonit.lan:8080/prometheus (Grafana appends /api/v1/query itself).
  3. HTTP headers: add one, Authorization = Bearer dmt_…. Grafana stores it encrypted and never shows it again.
  4. Save & test — Grafana reports the data source is working, and every dumbmonit_* series is then available in PromQL, with autocompletion on metric and label names.

Only the read routes are relayed — query, query_range, series, labels, label values, metadata and the version Grafana asks for when testing the connection. Anything else, admin/tsdb/delete_series first among them, answers 404: this entry point cannot change or erase anything.

If you already run a Prometheus, the other path works just as well: Prometheus federates DumbMonit with the job above, Grafana reads Prometheus, and the queries are identical.

Everything GET /metrics exposes. Values are read when you scrape; the _total counters are monotonic and reset only when the server restarts (dumbmonit_uptime_seconds tells you when that happened).

MetricTypeMeaning
dumbmonit_build_info{version}gaugeAlways 1. The version label is the running server version, the same one /api/health reports.
dumbmonit_uptime_secondsgaugeSeconds since the server process started.
dumbmonit_scheduler_cycles_totalcounterScheduler cycles run. The loop ticks once a second, so this doubles as a liveness signal.
dumbmonit_scheduler_cycle_secondsgaugeDuration of the last cycle. Normally under a millisecond: the cycle only dispatches, it does not wait for probes.
dumbmonit_scheduler_backloggaugeMonitors still due for a probe when the last cycle ended. Zero on a healthy instance; a lasting non-zero value means DUMBMONIT_MAX_CONCURRENT_PROBES is too low for the number of monitors and their intervals.
dumbmonit_probes_total{kind}counterProbes run, by monitor kind (snmp, proxmox, http…). Includes the ones a relay agent ran on the server’s behalf.
dumbmonit_probes_failed_total{kind}counterProbes that failed, by kind — an unreachable monitor or a configuration error. rate(dumbmonit_probes_failed_total[15m]) / rate(dumbmonit_probes_total[15m]) is the failure ratio.
dumbmonit_samples_written_totalcounterSamples accepted by VictoriaMetrics.
dumbmonit_sample_writes_failed_totalcounterBatches VictoriaMetrics refused or did not answer. The batch stays buffered and is retried; a rising value with a rising dumbmonit_samples_pending means the store is down or full.
dumbmonit_samples_pendinggaugeSamples still in the write buffer after the last flush. Normally 0.
dumbmonit_samples_dropped_totalcounterSamples dropped before even reaching the write buffer, because it was full or closed. Never retried — a rising value during an outage means data loss, not just delay.
dumbmonit_alerting_cycles_totalcounterAlerting cycles run (one every DUMBMONIT_ALERT_INTERVAL_SECS, 30 s by default).
dumbmonit_alerting_cycle_secondsgaugeDuration of the last alerting cycle, query time included. Approaching the interval means the rules are querying more than the store can serve.
dumbmonit_alerting_rules_evaluatedgaugeRules evaluated in the last cycle.
dumbmonit_alerting_rules_failedgaugeRules whose query failed in the last cycle. Their alerts are frozen, not resolved. Anything but 0 deserves a look at the logs.
dumbmonit_alerts{phase}gaugeAlerts the engine tracks, by phase: ok, pending (condition true, for not elapsed), firing, resolved.
dumbmonit_alerts_suppressedgaugeAlerts hidden because a parent monitor is down (the suppressed effective phase).
dumbmonit_alerts_silencedgaugeAlerts covered by a maintenance window.
dumbmonit_alerts_learninggaugeBaseline alerts still learning, and therefore silent.
dumbmonit_notifications_total{kind}counterNotifications sent, by channel kind (email, discord, ntfy…).
dumbmonit_notifications_failed_total{kind}counterNotifications the channel refused, by kind. They are requeued and retried; a steadily rising value is a broken webhook or SMTP account.
dumbmonit_agentsgaugeMachines that have registered an agent.
dumbmonit_agents_stalegaugeAgents that have pushed nothing for five minutes.
dumbmonit_database_upgauge1 when SQLite answers — the same check as /api/health.
dumbmonit_database_bytesgaugeSize of dumbmonit.db and its write-ahead log. Does not include /data/vm.
dumbmonit_victoriametrics_upgauge1 when VictoriaMetrics answers — the same check as /api/health. 0 while the embedded child is restarting.
dumbmonit_victoriametrics_embeddedgauge1 when this server runs VictoriaMetrics itself, 0 when DUMBMONIT_VM_URL points at an external one.

No metric on this page carries a per-monitor label: the document is the same size for one monitor and for a thousand, and no monitor name leaks to whoever can scrape it.

Two alerts worth adding on your side, once DumbMonit is scraped:

# The monitoring stopped monitoring
up{job="dumbmonit"} == 0
# Measurements are piling up instead of being stored
dumbmonit_samples_pending > 0 and increase(dumbmonit_sample_writes_failed_total[10m]) > 0

DUMBMONIT_METRICS_PUBLIC=true serves /metrics, /federate and /prometheus/api/v1/… to anyone who can reach the port, for people who filter the port themselves — anyone on that network can then read every measurement of every monitor and the state of the instance, so only set it when something else already restricts who reaches the port.