Operations

OxiMail is one binary, one systemd unit, one SQLite database, and a blob directory. Day-to-day operations are correspondingly small: read journald, scrape one metrics endpoint, keep backups, and swap the binary to upgrade.

Logs

The server logs to stdout and journald captures it — there is no /var/log/oximail:

journalctl -u oximail -f              # follow
journalctl -u oximail --since -1h     # recent window

The level is raised for a bounded time, never until further notice: oximail log elevate --directives oximail_imap=debug --duration-secs 600 (or PUT /admin/v1/logs/level) and it reverts on its own; the elevation and the return are both journal events. A lasting change goes in the TOML, where it stays visible. The raw exchange of one account or one peer is captured by oximail log trace, scoped, bounded, redacted and purged on stop (CLI, admin API). Logging uses structured tracing fields (account_id, tenant_id, method, error), and log messages describe what actually happened, not what was intended — a hard project rule. Diagnostic context is deliberately attributable: an IMAP session probing a nonexistent folder logs the account and peer address, a SQLITE_BUSY names the table and operation involved.

What a line records

At info, the journal records events, not exchanges (ADR-140):

An outbound message is one queued line (sender, recipient count, size, origin), then one delivered line per recipient, in the Postfix shape: the relay reached (host[ip]:port), the local address it left from, the TLS version, the attempt, how long it waited in the queue, and the peer’s reply. The “attempting” and “connected” lines, and the post-quantum report of each delivery, are debug (the latter is also a counter).
An inbound message is one message accepted line that carries its spam verdict (spam_score, flag_junk) and whether the session was TLS; the separate spam and STARTTLS lines are debug.
A WebSocket session writes its closing line with its duration and cause; an IMAP session’s closing line names the client from its ID command.
A CalDAV or CardDAV sign-in writes one line per device and per hour: the account, the secret used (an app password’s id, or the account password), the client address and the client the request announced. IMAP and SMTP already say as much per session, so an administrator can tell one device from another on every protocol, and see which device still uses the account password.
A repeated state is written once. A Web Push subscription that matches no open tab is reported once per subscription at info, then at debug; the access to a shared account is written once per session and per pair (grantee, owner, collection, level), and again when the level changes, while refusals keep a line each time and oximail_jmap_shared_access_calls_total still counts every call. A client that presents an IP address as TLS SNI is no longer logged by rustls.
The rate limits in force are named. The boot line gives the [rate_limit] TOML values as http_per_ip_baseline and http_per_account_baseline; each application of the stored rate_limit.* overrides (at boot and on PUT /admin/v1/config) then logs rate-limit overrides in force with field=effective (baseline b) for every override that differs from the TOML, or none in force.

TLS handshake logs

The four listeners that terminate TLS — IMAPS, IMAP STARTTLS, SMTPS, SMTP STARTTLS — separate two events that used to share a single warning:

A peer that left before negotiating anything. A TCP availability probe, a port scan, a client that gave up. This is not a failure and is no longer logged as one: it drops a level, keeping its peer and error text, and RUST_LOG=debug still restores every byte of it.
A real negotiation failure. No cipher suite or group in common, a malformed message, an alert received, a certificate refused. This stays a warning, because it is one.

Both branches carry a stable reason field. Key your alerting on that field rather than on the message text. The reason this distinction earns its place: one production instance was emitting 480 of those warnings a day from a single monitoring probe that opened 993 every three minutes and hung up. A warning that fires 480 times a day perfectly harmlessly is not a cosmetic annoyance — it trains everyone to ignore the whole category, and a genuine handshake failure then hides inside it at the same port, with the same message, at the same level. That matters most right after a change to the negotiated groups, whose failure mode is exactly a handshake that stops completing with one incompatible client. The two explicit-timeout branches (120 s on IMAPS, 30 s on STARTTLS) remain warnings: a peer that holds a connection for two minutes without concluding is rare and genuinely worth a look.

Metrics

A Prometheus endpoint is served on the admin surface at /metrics. Give it its own token: [admin] metrics_token_file = "/etc/oximail/metrics.token" (a sidecar file, mode 0600, one line, never an inline key, since the TOML is backed up and diffed) makes /metrics accept that read-only bearer and, once set, the administration token stops opening /metrics on this box for good, even if the file is later removed. Before that setting a Prometheus scrape configuration held a token able to read mail, change the configuration and lift bans. doctor presents the metrics token to its own probe, and reports a refusal as a degradation rather than skipped. Gauges worth alerting on:

Metric	Alert when
`oximail_storage_db_bytes`, `oximail_storage_wal_bytes`	WAL grows past the database size — the checkpoint worker should keep it under ~64 MiB.
`oximail_erasure_jobs_stuck`	`> 0` — an erasure is parked on a durable failure and needs an operator.
`oximail_smarthost_ip_listed{ip,list}`	`> 0` — your egress IP is on a DNSBL (outbound relay).
`oximail_dane_tlsa_state{host,state}`	`state="mismatch"` at 1 — the TLSA published for `_25._tcp.<host>` no longer matches the served certificate, validating senders reject; `state="probe_error"` at 1 for a couple of hours — the hourly sonde cannot validate the answer (resolver, DNSSEC), which is not silence. `absent` is a posture, not a fault. Exactly one state is 1 per host.
audit-retention last-sweep gauge	stale — the retention worker stopped sweeping.
`oximail_log_events_total{level,target}`	a target’s WARN or ERROR count climbing — which module is producing a storm, without reading the journal. Bounded cardinality: the target is a fixed string per call site.
`oximail_worker_backlog{worker}`	a background worker’s remaining debt not descending — a stuck worker and an idle one used to draw the same curve.
`oximail_at_rest_volume`	`plain` on a production box (encryption at rest).
`oximail_access_log_append_failed_total`	`> 0` on a `best-effort` tenant — reads served unlogged (compliance).
`oximail_federation_grants_expiring{tenant}`	`> 0` — federation grants about to lapse, their subject warned fourteen days ahead or not (sharing).
`oximail_federation_mirrors{state}`, `oximail_federation_door_requests_total{class,outcome}`	mirrors staying in `unreachable` or `copying` — a paired organization’s shared folders no longer reach this server; refusals climbing in the door’s outcomes. No federation metric carries a peer label: `oximail federation peers list` names the peers on the box (federation).
`oximail_search_index_state{collection}`	`> 0` for longer than a rebuild: `1` the full-text index is rebuilding, `2` it cannot be opened and searches run on a degraded fallback (the search index).
`oximail_calendar_recurrence_out_of_profile_total{face,reason}`	climbing — a client writes recurrence rules outside the closed profile, stored as received for now; `oximail_calendar_get_occurrences_rule_form_total{form}` shows which form of rule (`legacy_inline`, `id`, `rfc8984`) `Event/getOccurrences` still receives (calendar).

Where the time of a write goes. Five histograms (buckets 0.5 ms to 30 s), published from boot, show what a client waits for; no alert rule reads them, they are there to measure before a storage setting is changed:

Histogram	Measures
`oximail_sqlite_write_lock_wait_seconds{result}`	The wait for SQLite’s write lock (`BEGIN IMMEDIATE`, inside which `busy_timeout` sleeps), `acquired` or `failed`. There is no writer queue: every pooled connection takes the lock itself.
`oximail_sqlite_write_transaction_seconds{outcome}`	How long a write transaction holds the lock, to its `commit` or `rollback`.
`oximail_sqlite_commit_seconds`	The `COMMIT` statement alone, the part a change of `synchronous` moves.
`oximail_sqlite_autocommit_write_seconds{result}`	A statement that writes outside any transaction: SQLite takes the lock, writes and commits in that one call, so this is the whole implicit transaction, `ok` or `failed`. Reads, and writes inside a transaction, are not counted.
`oximail_imap_command_duration_seconds{command,result}`	An IMAP command, from its parsed line to its tagged reply; `result` is `OK`, `NO`, `BAD`, or `none` when the session ended first. `IDLE` is not observed (its length is the client’s).
`oximail_smtp_data_reply_seconds{listener,class}`	From the end of a message’s data to the reply decided, `inbound` or `submission`, `2xx`, `4xx` or `5xx`.

oximail_durability_barrier_seconds{door,result} measures the sync that runs before a reply promising the data is on stable storage (storage). Every label is a closed set: no address, account, path or mailbox. Per-organization metrics can gain a domain label ([metrics] config) — leave it off unless you need the breakdown; the cardinality cost is real. Two labels are closed sets by construction. The method label of oximail_jmap_requests_total is a registered method name or unknown: a method name a client invents makes no series of its own. The client label of oximail_client_requests_total (the client mix) counts only the requests of mail, calendar and contacts clients, on /jmap, /caldav, /carddav and their discovery documents, never a Prometheus scrape or a health probe; a web browser is Browser, OxiMail’s own tools are CLI/API, and each authenticated IMAP session counts once, at its end, as imap:<family> read from its ID command (imap:Unknown without one), never summed with HTTP requests.

The background workers

The binary runs its maintenance internally — there is no external cron to install:

Delivery queue worker — outbound retries with backoff, DSN generation (SMTP layer).
Search index worker — batches Tantivy commits; commits on quiescence. Its catch-up indexes what is due, 256 rows a commit with a 2-second read budget, so a live write never waits long behind a commit; it is also what rebuilds the index after an upgrade (the search index).
WAL checkpoint worker — publishes size gauges, TRUNCATE-checkpoints past 64 MiB.
Fail2ban cleanup — expires bans every few minutes.
Reindex worker — converges the reindex queue; terminal conditions resolve entries instead of retrying forever. It selects nothing while no index is open, so the marks waiting for the index are not consumed in one tick.
Relay domain re-verification — re-checks the DNS proofs of relay and envelope-sender domains two minutes after start, then every six hours; a proof suspended, or not re-verified for 72 hours, defers the mail it covered with 451 4.7.1 until it is proven again (outbound relay).
Erasure maintenance — adopts and resumes erasure jobs whose driver died; boot-time orphan sweep.
Audit retention — purges audit rows past the horizon.
Alert schedulers — fire calendar and task reminders.

Backups

oximail backup   --config /etc/oximail/oximail.toml ...
oximail restore  ...
oximail verify-backup ...

A backup must capture both the SQLite database and the blob directory (and /etc/oximail, which holds the config, DKIM keys, and the root key material: it no longer guards content, but it wraps secrets valid elsewhere, app-password hashes and provider API keys among them). The database in a backup is plain SQLite and the blobs are in clear, so the copy that leaves the box must itself be encrypted: the hypervisor backup, client-side encrypted, is the backup of record (encryption at rest). verify-backup checks a backup’s integrity before you need it. Test a restore on a scratch machine, not during an incident.

Without a hypervisor: the encrypted repository

A box whose hypervisor does not take snapshots (a VPS at a hosting provider, a Raspberry Pi, a small office box) backs itself up with oximail backup --to <repository>. The repository holds age-encrypted objects — every attachment, the database, the configuration directory, the key escrow, the erasure ledger — and one snapshot per run that lists them.

The box writes backups it can never read. It holds only the public keys of [backup] recipients (age1…); the identity that opens them, written once by oximail backup key generate --out <file> --apply, is moved off the box. Several recipients each open the backups on their own.
Incremental and resumable. Attachments never change (they are named by their content), so one already in the repository is not sent again; a run writes its snapshot last, and an interrupted one resumes where it stopped.
Authenticated. age does not say who encrypted a file, so each snapshot carries an HMAC under a key of the box, whose copy is in the identity file: a forged or altered snapshot is refused. --json reports the snapshot’s SHA-256; record it outside the box, and restore --expect-sha256 refuses any other.
The box never deletes. A ransomware with root on the box can do what the box can do: give it a destination it can only add to, and run oximail backup prune --repository <dir> --identity <file> (retention per day, week and month, dry-run by default) where the identity lives.
Readable without OxiMail. Every file opens with the age tool: age -d -i identity.txt snapshots/<name>.age gives the list of objects in JSON lines, age -d … objects/ab/ab12… an attachment, age -d … | gunzip the database.

oximail backup --to /mnt/backup/oximail --json                    # nightly
oximail verify-backup --from /mnt/backup/oximail --identity id.txt --full
oximail restore --from /mnt/backup/oximail --identity id.txt --expect-sha256 <hash>

restore authenticates the snapshot and checks every object before writing anything, merges the erasure ledger, freezes the erased accounts that came back, and leaves the configuration in restored-config/ for review (--with-config writes it in place). What the repository shows to whoever holds it: the number and size of the objects and when new ones arrive; the names and contents are encrypted. The old --output --encrypt (a passphrase, everything read into memory) is kept only so existing backups restore. Into a bucket. --to s3://bucket/prefix, with [backup] s3_endpoint, s3_region, s3_access_key_id and s3_secret_access_key_file (0600). The bucket must be created with Object Lock and a default retention of at least 7 days, and the box’s key may only add and list. Before each run the box checks both, on the bucket itself: versioning and the lock are read, then a probe object must be impossible for the box to delete and to read. Anything else and the run writes nothing. Without versioning, “add only” means nothing — a write over an existing name replaces it — so a provider without Object Lock is refused; keep a directory there, pulled by another machine. Off the box, restore, verify and prune take the operator’s key from the standard AWS_* variables; prune’s removals are delete markers (add a lifecycle rule for noncurrent versions), and it extends the lock of everything still kept. COMPLIANCE or GOVERNANCE. Both lock modes are accepted. A lock in GOVERNANCE mode holds against the box, whose key cannot delete (the probe proves it), and lets an operator undo a mistaken retention, which COMPLIANCE makes final. But a key holding the bypass permission can remove it, so a run into such a bucket says so every time: a WARNING line, the log, and a warnings field in --json. The run record keeps the lock mode, and doctor backup_repository repeats it in its detail without turning the check into a standing warning. COMPLIANCE is the recommended mode; under GOVERNANCE, the key that runs prune never holds s3:BypassGovernanceRetention. The locks count time by the storage’s clock. Every request carries the box’s time and the storage refuses a gap above 15 minutes, so a skewed clock already stops the run; that refusal (RequestTimeTooSkewed) now names the two clocks and says the locks depend on the storage’s. A compromised box cannot rewrite the history. The key that authenticates snapshots lives on the box, so once the box is taken, forged snapshots are authentic. Snapshots are therefore dated by the repository’s clock, never by the date they carry. A snapshot that says it was made after the repository received it, that shares its sequence, or that was written twice is suspect: prune removes nothing and names it, and restore never takes it unless it is named with --snapshot. Prune removes an unused object only after seeing it unused at two passes a grace period apart, and never one received after the newest snapshot. One account back. oximail account import --from-backup <repository> --identity <file> --account <address> [--into <address>] [--apply] brings back one account’s mail, contacts, events, tasks, Drive files and notes beside what it holds, under a folder named after the snapshot’s day (Restored 2026-09-20): nothing is replaced, and what the account still holds is not brought back twice. --into writes into another account (a colleague who left). Refused, with nothing written: an erased account, an address that belonged to another account in the snapshot, another tenant, a frozen account. Filters, shares and rights are not restored; a message naming an erased person is redacted first; quota is charged. Dry-run by default. A chat room comes back whole, not per account: oximail chat restore-room --room <id> --from-backup <repository> --identity <file> [--apply] takes the room’s database rows and attachments out of the repository. A restore drill that doctor sees. verify-backup --from <repository> --identity <file> --full --record, where the identity is, decrypts every object and, when nothing is wrong, leaves a dated receipt in the repository. The box lists the receipts with its own key (it cannot read them), and doctor backup_restore_drill warns when none is newer than 90 days. An object above 256 MiB goes to a bucket in parts of 64 MiB. Rate and duration. [backup] max_rate_mbps caps what a run sends; max_duration_minutes stops it between two objects, without a snapshot, and the next run continues (run the first upload without it). doctor backup_repository reads the repository with the box’s own key: the last snapshot the box wrote must still be there, and a first upload still unfinished after a day is a failure — nothing can be restored yet. The plan is declared, and doctor holds you to it. The setup wizard asks how the box will be backed up and writes the answer as [backup] strategy ("hypervisor-snapshot", the default, "oximail-backup" or "none"). doctor backup_plan fails on a box with no plan or "none", and on an oximail-backup plan that has never run; a run older than seven days is a warning. A hypervisor snapshot is invisible from inside the box, so the hypervisor says so: its vzdump hook, at backup-end, runs oximail backup mark --source hypervisor --apply, which records the date and the source in the marker backup_plan reads (--source external for another tool on the host). Without a mark the check is a warning, “declared at setup, never confirmed from outside”: a declaration nothing ever confirmed is not a backup; with one it shows the date of the last confirmation, and a confirmation older than seven days is a warning (the hook or the job stopped). An oximail-backup plan is judged on the verb’s own record only: a hypervisor mark does not stand in for it. The escrow restores the same box. The recovery escrow of the root key also carries the instance secret, the seed of every bearer token, so a box restored on new hardware from its escrow is the same box and keeps its sessions. Each instance records a stable identity; doctor instance_identity fails when the secret changed since the database was created (tokens issued before no longer validate), and names the two ways out: put the original secret back from the escrow (oximail key recover), or accept the change (oximail key adopt-identity --yes). An escrow made before this carries the root key only: doctor server_key_escrow warns where that matters (a box whose key was rotated), and oximail key upgrade-escrow --yes rewrites it under the same recovery key (encryption at rest). oximail backup --tenant <id> narrows a backup to one organization. Two properties of that mode are worth knowing, because both were once quietly untrue:

It carries the organization’s attachment blobs, not only its rows. A per-organization archive whose messages restore without their attachments verifies clean while being unrestorable in the only sense that matters.
The set of tables it filters is derived from the schema and guarded, not a list maintained by hand. A hand-maintained list goes stale the moment a table is added, and the failure is silent in the worst direction: rows belonging to other organizations riding along in a tenant-scoped export.

Upgrades

An upgrade is a binary swap; there is no separate migration step to run:

systemctl stop oximail && mv oximail.new /usr/local/bin/oximail && systemctl start oximail

At boot, pending schema migrations apply automatically in one exclusive transaction; a failure rolls everything back and the server refuses to start rather than running half-migrated. Verify the deploy afterwards:

curl -s -H "Authorization: Bearer $ADMIN_TOKEN" https://mail.example.com/admin/v1/version
# → version + migrations_applied

oximail status, oximail health, and oximail doctor provide local checks (process, ports, config, storage) without the HTTP round-trip. Three guards stand between a binary and a database it should not touch:

An older binary never runs on a newer database. The ADR-104 boot guard now also covers every verb that opens the database directly (sql, reindex, search purge-orphans, chat rekey, principal reconcile, doctor), and oximail schema check is that guard as a verb: exit 0 when this binary accepts the database, 2 when the database is newer or unreadable, read-only, never migrating. A migration namespace the binary does not know (a base an extension migrated) is a boot refusal naming the namespace, and schema check answers "verdict": "unknown-namespace".
restore refuses a backup made by a newer binary, and the manifest carries the core’s schema version for that judgement, since the previous migration_version was the maximum across every namespace.
A rollback is proven before the swap, by the target binary. The deploy path runs schema check with the previous binary on every host before swapping back, instead of discovering the refusal at the next boot; relay-spool reabsorb no longer touches the schema to make an old binary accept the base.

The configuration file is a contract: an unknown key is an error at load, a repeated block nests ([[smtp.dkim_keys]] is the convention, defended by a test), and doctor, check-config and every other verb load the extension sections the server itself reads. check-config also refuses a loopback listener behind a reverse proxy it does not recognise (every visitor would share one rate-limit bucket) and a backup MX without a relay spool. setup verify finally exits non-zero (0 all good, 1 degraded, 2 a failing check), says who renews the certificate (cert_strategy, renewal_owner) and what the box must reach; the wizard publishes the server’s IPv6 (AAAA, ip6: in SPF) and no longer mistakes a v6 address for a v4. No CLI verb requires a secret on its command line any more: every token takes a --…-file form, since an argument is readable in the process list and the shell history. SQLite allocates in jemalloc, in an arena of its own. The server’s allocator has been jemalloc since June, but SQLite is C and had never left glibc, whose per-thread arenas keep freed memory; measured on a production box, jemalloc carried 0.3 % of the heap. The oximail_jemalloc_* gauges now describe the heap they name. The first start after an upgrade may rebuild the search index. When a release changes the index’s generation, the server sets the old index aside, serves a new one seeded from it, and recomposes every document in the background; the listeners do not wait, and doctor search_index shows the progress (the search index). An older binary refuses the newer index, as it refuses the newer database. Local snapshots have a retention policy, and the deploy path applies it. After a swap is verified active, the two most recent .bak binaries survive and any database snapshot older than 14 days is removed, hand-placed one-offs included. A host whose swap failed keeps everything, because that is rollback material. Two things this is not: it is not your backup — the durable layer stays the off-box backups — and it only touches data.db*.pre* snapshots, never the live database. Left unmanaged this grows quietly: one box had accumulated 85 GB of rollback binaries, and a backup MX was at 89 % disk.

Graceful shutdown

systemctl stop (SIGTERM) triggers a drain: workers finish their in-flight items within a bounded drain budget, the search index flushes, and the WAL is checkpointed. Restarts normally complete in the time the workers actually need — typically near-instant. A “drain cap reached” warning in the logs means a genuinely slow worker and is worth investigating. A restart no longer accepts a connection it then cuts. At the stop, the SMTP, IMAP and HTTPS listeners stop accepting at once, so the kernel refuses what comes next and the client retries. A session waiting for its next command is told so: 421 4.3.2 on SMTP, * BYE [UNAVAILABLE] on IMAP (in IDLE and before STARTTLS too). A session in the middle of a transaction finishes it: a DATA or BDAT in progress is never cut by the stop, and an HTTPS connection the box terminates itself finishes its request and closes. The drain waits for the workers and for the open sessions within the one budget; when it runs out, the warning names the workers and sessions it was still waiting for. Before, the listeners kept accepting during the whole budget (up to 10 s on a busy box) and then cut those connections, and a message stored but cut before its 250 could be sent twice.

Security operations

The three-layer defense (kernel nftables, in-binary rate limiting + fail2ban, monitoring) is described in the security model; the operational surface is:

oximail unban <ip>                    # lift a fail2ban ban
oximail ban list|add ...              # inspect and manage bans
curl .../admin/v1/bans                # the same over the admin API

[security] trusted_ips (CIDR supported) bypasses both the rate limiter and fail2ban for your own infrastructure. Do not install the fail2ban daemon next to OxiMail — the server tracks its own auth failures across every protocol, with the real client IP even behind a proxy. Behind a NAT router. A box on a private network, reached through port forwarding (DNAT) on the router, works as a direct box: the router rewrites the destination, the source address of each client arrives intact. Two router behaviours break that, and both concern trusted_ips. With hairpin NAT, a computer of the LAN that opens the public name is sent back to the box under the router’s own LAN address. Some routers also rewrite the source of every forwarded connection, clients from the Internet included. If the router’s address falls in a trusted_ips range (the usual 192.168.0.0/16), everything it relays that way escapes the rate limit and fail2ban; if it does not, one ban on the router locks everybody out. oximail doctor reports, under trusted_ips_gateway, a gateway of the box’s routing table that falls in trusted_ips. Narrow the range to exclude the router, and check that the router keeps the real source address (the addresses in the box’s log are the measure).

The doctor round

oximail doctor is the composite check: exit 0 all good, 1 degraded, 2 a failure. Each check comes out ok, warn, fail or skipped, and skipped means nothing was measured, never a pass. --json is an array of {check, status, detail} objects, to which three fields were added without changing those: family (mail, backup, storage, security, accounts, system, or extension for a check an extension contributed), facts with the dated facts as UTCDate (observedAt of the last backup or recovery drill, expiresAt of the certificate), and, on a warn or a fail only, remedy, the verb to type. A field with nothing to say is absent, not null.

{"check": "backup_plan", "status": "warn", "detail": "hypervisor snapshot last confirmed 9 day(s) ago …",
 "family": "backup", "facts": {"observedAt": "2026-09-16T02:10:00Z"},
 "remedy": "oximail backup mark --source hypervisor --apply"}

The round is recorded every 15 minutes. oximail-health.timer runs oximail doctor --json --record as the service user; the stored round (30 days kept) is what HealthReport/query and HealthReport/get serve over JMAP (admin API). The setup wizard writes the two units; on a box installed before them, oximail setup health-timer --apply writes them and enables the timer. --record is refused as root, and its exit code is the recording’s (0 stored, 2 not stored). A tenant admin reads the report of a box where its tenant is alone; on a box with several tenants only an instance operator does, an account made so at the console alone (oximail admin instance grant --account … --by <name> --apply). instance_admin warns when a box with several tenants has no active operator, and names a holder whose row opens nothing (the account demoted, disabled or without a passkey). --reachability adds the network probes, which a diagnostic run during an outage must be able to skip: without it they say skipped. dnsbl_self asks the public DNS blocklists about the sending address ([metrics] dnsbl_self_ips, else the detected public IPv4): listed is a warning, a query the list refused (a public resolver) is a warning too, because the state is then unknown. meet_join_origin checks that meeting links open a page; when the host of web_base_url resolves to the box itself behind a declared proxy, the probe would only hit the box, so it is inconclusive (skipped) and the detail gives the curl to run from outside. A few checks worth knowing by name:

Check	What it says
`server_key_escrow`	`fail` until the root key is escrowed; `oximail key escrow --guided` performs the ceremony.
`account_access_expiry`	Operator grants on another account expiring within 14 days or expired, and grants without a date on a frozen owner. A grant to a service account (`account create --service`) is not a departure: it leaves those counts, and one that carries a date is named as an anomaly.
`backup_plan`, `instance_identity`	The declared backup plan and its last confirmation, the instance secret (backups).
`telemetry_consent`	`warn` when no answer to the reports question is recorded: nothing is sent (community telemetry).
`orphaned_rows`	Per-account rows whose account does not exist, by table, with the count and the first owners. Report only. Owners without an account by design (the change axis of distribution groups, the imported spam corpus, DMARC reports in the queue) are not reported.
`directory_closed_accounts`	Closed accounts whose card is still in the people directory, so in every colleague’s address book (closed before the freeze withdrew it, or a withdrawal that failed and was logged); `oximail admin profile backfill --apply` withdraws them.
`erasure_held_residue`	Copies of erased people’s mail a colleague’s legal hold still keeps unredacted, and when the first hold ends (compliance); `warn` when a hold has ended and the copy is still unredacted, which the running server’s erasure worker does at its next tick.
`erasure_ledger`	The erasure ledger against the database (compliance); `fail` when an erased account is present again (a restore brought it back; remedy `oximail erasure replay`) or the ledger does not read; `warn` when an erasure is newer than the last known backup and no `[erasure] ledger_hook` is set, or the hook has not shipped the latest lines.
`received_at_format`	Messages whose `receivedAt` is not in the RFC 3339 UTC form (an old import); they sort out of date order until `oximail email normalize-received-at --apply`.
`journal_retention`	The journald window actually kept against `MaxRetentionSec`; when it is short, the occupancy against `SystemMaxUse` and the daily disk throughput say why: a cap filled by noise, or history not accumulated yet.
`sender_domain_proofs`	`fail` when a proven relay or envelope-sender domain is suspended or not re-verified for 72 hours (its mail is deferred), `warn` when it missed a re-verification (none for two six-hour intervals); the remedy is `oximail account relay verify` or `oximail account envelope-domain verify` (outbound relay).
`smime_trust`, `bimi_trust`	`skipped` on a backup MX or a smarthost, where no mailbox is stored and neither worker runs.
`search_index`, `search_index_leftovers`, `search_text_evidence`	The full-text index, probed read-only: open, rebuilding, of an older generation, empty or unopenable, with the rows still due and the removed documents still on disk; directories beside it that are not the index; messages whose text survived only in the old index (the search index).
`recurrence_rules`, `recurrence_profile`	`warn` on events or tasks carrying more recurrence rules than an object may, which every write path refuses (rows from before the guard, served as their master alone), and on stored rules outside the closed recurrence profile, counted by reason before the server refuses them at write (calendar).

Community telemetry

OxiMail can send its publisher a short report on how the server runs (ADR-153). Nothing is sent without the operator’s answer. The setup wizard asks Send reports? [y]es / [c]rashes only / [n]o / [d]etails with no answer preselected (Enter alone asks again, d shows what is sent), on a backup MX too; a setup.toml must carry [server] telemetry = "yes" | "crashes" | "no" or the run stops and names the key. The answer is written in a consent marker in the data directory, next to the installation id, and a box without a marker sends nothing. [telemetry] enabled = false switches the channel off whatever the marker says.

oximail telemetry status [--json]              # what left the box, byte for byte; exit 1 when the channel cannot work
oximail telemetry enable [--crashes-only] --apply
oximail telemetry disable --apply              # a recorded "no" stays an answer
oximail telemetry purge --apply                # the receiver deletes what this installation sent

enable and disable show what they would record and write only with --apply; they take effect within the hour, without a restart. enable also re-arms a channel the receiver withdrew. A report is an envelope (report format, the window it covers, the installation id, the version, the consent text version and the categories consented to), plus the part of each category consented to:

crash: how many runs did not stop cleanly, how systemd saw them end, and where OxiMail’s code panicked (crate, file, line; never the message or a value);
usage: OS, architecture and distribution family from a closed list, whether the build is official or custom, the role, the features on, uptime, accounts and mail as ranges, the share of spam rejected, the five most called methods with the share of their calls in each latency bucket, the ratio of the p95 latency to the one measured on the previous version, error types, protocols and client families in use, configuration changes and how far the setup got (closed lists). No mail content, no address, domain, hostname or licence key.

A report leaves fifteen minutes after start, then once a day ([telemetry] interval_secs), and when the server stops cleanly, which sends the window it holds (a window shorter than a minute is dropped). Reports go out over https only (plain http to a loopback address alone) and never follow a redirect. The installation id is derived from a random secret in the data directory: the value the receiver stores is not shown on the box (telemetry status --show-id or telemetry id --show-id print it), the id is no longer written to the journal at start, a data directory restored or cloned onto another machine gets a new secret, and purge presents the secret itself, then rotates it. Every field of each report format is listed at oximail.ch/privacy/telemetry, which the generated configuration points to; the consent text states the retention (90 days) and who receives the reports.

When something is wrong

journalctl -u oximail --since -1h — the failure is loud by design; look for error-level lines with context fields.
oximail doctor — config, storage, and port diagnostics.
/metrics — is a queue growing, a worker stuck, a gauge stale?
For delivery problems, the DNS checklist and the queue CLI (oximail queue …) come next.

If legitimate mail is landing in spam, check the clock. oximail doctor reports clock health, and it is the second thing to look at after DNS. An inbound DKIM signature carrying a timestamp is refused when it arrives more than 300 seconds in our future, so a box whose clock lags starts rejecting signatures on perfectly good mail — which then scores as spam. The symptom looks like a DNS or DKIM problem and never looks like a clock. Note the exposure is one-directional: our own outbound signatures carry no timestamp, so a recipient has nothing of ours to judge, and nobody ever complains to us about it.

Get Started

Operator Manual

Developer / JMAP API

Architecture Reference

Reference

Logs

What a line records

TLS handshake logs

Metrics

The background workers

Backups

Without a hypervisor: the encrypted repository

Upgrades

Graceful shutdown

Security operations

The doctor round

Community telemetry

When something is wrong

​Logs

​What a line records

​TLS handshake logs

​Metrics

​The background workers

​Backups

​Without a hypervisor: the encrypted repository

​Upgrades

​Graceful shutdown

​Security operations

​The doctor round

​Community telemetry

​When something is wrong

Logs

What a line records

TLS handshake logs

Metrics

The background workers

Backups

Without a hypervisor: the encrypted repository

Upgrades

Graceful shutdown

Security operations

The doctor round

Community telemetry

When something is wrong