oximail-store

oximail-store is the storage core every domain crate depends on. It defines three storage traits — Store (structured metadata), BlobStore (binary content), SearchStore (full-text) — and ships the default implementation behind each: a connection-pooled SQLite database, a content-addressed filesystem blob store, and a Tantivy full-text index. It also owns the schema: the versioned migration set that is applied automatically at boot.

​

Overview

Domain crates never talk to a database driver directly. They hold an Arc<dyn SqlBackend> and build typed Param slices; the backend translates those to the native driver and returns OwnedRow values that own their data. This is the boundary between the SQL driver and the rest of OxiMail — no crate above oximail-store ever imports rusqlite. The crate sits at the very bottom of the dependency graph. oximail-core re-exports its Id, BlobId, and State primitives; every other crate reaches storage through the traits defined here. The contracts are deliberately narrow: every method on Store, BlobStore, and SearchStore takes tenant_id as a mandatory parameter, so a handler physically cannot read another organization’s data even with a bug. The community (AGPL) build is SQLite + Tantivy + filesystem. There is one SqlBackend implementation in the crate — SqliteBackend. That is the storage core OxiMail ships and runs in production.

​

Features

• SqlBackend trait — a type-erased SQL interface (execute, query, query_one, query_opt, begin, execute_batch) implemented by SqliteBackend. Domain stores build &[Param] slices and read OwnedRow results instead of touching the driver.
• r2d2 connection pool, no Mutex<Connection> — SqliteBackend checks a connection out of an r2d2 pool per call. WAL mode allows concurrent readers while a single writer holds the lock.
• Per-connection PRAGMAs — every pooled connection is initialized with journal_mode=WAL, busy_timeout=5000, synchronous=NORMAL, cache_size, and foreign_keys=ON, so concurrent JMAP requests do not see spurious SQLITE_BUSY errors.
• Versioned migrations (ADR-013) — a static, ordered migration set applied at boot inside a single BEGIN EXCLUSIVE transaction. Fail-loud: any failure rolls the whole batch back and the server refuses to start.
• FK consistency validator — a compile-/test-time check (validate_fk_consistency) that catches a class of foreign-key bug SQLite accepts silently at CREATE TABLE time but only surfaces at the first parent DELETE.
• Store trait — structured CRUD, JMAP change tracking (RFC 8620 §5.2), optimistic concurrency (record_changes / atomic_set with ifInState), config overrides, and native soft-delete / undelete.
• BlobStore trait + filesystem store — content-addressed binary storage keyed by SHA-256 of the cleartext, with atomic write (file + metadata row in one operation).
• SearchStore trait + Tantivy index — full-text search with a batching background writer.
• Mandatory tenant isolation (ADR-015) — tenant_id is a required argument on every store method; every SQL query filters on it.

​

How it works

​

The backend abstraction

SqlBackend is the single seam between OxiMail and its database. A domain store holds an Arc<dyn SqlBackend> and calls execute / query / query_one / query_opt with a SQL string and a &[Param] slice. Param is a small enum (Null, Integer, Real, Text, Blob) with From conversions for every type used in a query, so call sites build parameters with the params![] macro rather than the driver’s own macro. Results come back as OwnedRow values. Unlike a borrowed driver row, an OwnedRow owns its data and can be returned from a method or stored in a collection. Rows can carry a shared column-name table, which lets a parser read columns by name and tolerate any subset or order — the substrate for column projection, where a SELECT omits columns the caller does not need. A missing mandatory column is a hard error; a missing optional column reads as None, exactly as a SQL NULL would. Nothing is silently coerced. A Transaction exposes the same operations and must be committed explicitly; it rolls back on drop. The SqlExecutor trait is implemented by both the backend and a transaction, so a store method can be written once and run either inside or outside a transaction — this is how /set handlers keep every write on the same connection (a hard rule: two BEGIN IMMEDIATE on two connections is a deadlock).

​

Pooling, not locking

The SQLite backend wraps an r2d2 pool (8 connections by default). Each call checks out a connection, runs the statement, and returns it — there is no Mutex<Connection> anywhere. WAL mode lets readers proceed while a single writer holds the write lock. Every connection acquired from the pool runs the same PRAGMA batch: WAL journaling, a 5-second busy_timeout, synchronous=NORMAL, a bounded per-connection page cache, and foreign_keys=ON. The busy_timeout is what keeps concurrent JMAP requests from seeing spurious SQLITE_BUSY errors; the backend also keeps a global counter of any busy errors that do occur, enriched from the calling tracing span, so the condition is observable rather than swallowed.

​

Migrations: versioned, auto-applied, fail-loud

The schema is a static slice, MIGRATIONS, of 136 numbered migrations (versions 1 through 136 on the current line; MIGRATION_COUNT is exposed via /admin/v1/version as migrations_applied for deploy verification). Each migration is a version number, a description, and a block of raw SQL. At boot, apply_migrations_backend runs every migration whose version is greater than the recorded MAX(version), inside a single BEGIN EXCLUSIVE transaction. The exclusive lock serializes concurrent instances (a rolling restart or a parallel boot blocks rather than races on schema_version). If any migration fails, the whole transaction rolls back and the server refuses to start — there is no manual upgrade step and no half-applied schema (ADR-013). After a schema change it also refreshes query-planner statistics with ANALYZE; an ANALYZE failure is logged but does not abort boot, because the database is fully usable, merely unoptimized. An out-of-tree companion crate can manage its own schema alongside core through apply_migration_set with a separate namespace, so the two version lines never collide.

​

The FK-consistency validator

SQLite accepts a FOREIGN KEY (a, b) REFERENCES x(c, d) at CREATE TABLE time even when (c, d) is not a unique tuple on x. The mismatch only surfaces at the first DELETE on the parent, as the runtime error foreign key mismatch. This is a real bug class: a past migration shipped an FK referencing (tenant_id, id) on a table whose primary key was (tenant_id, account_id, id) — caught only when a regression test deleted a parent row. validate_fk_consistency (fk_validation.rs) closes that gap. It replays the migration set in version order, maintaining a model of each table’s uniqueness sources (primary key, table-level and inline UNIQUE, CREATE UNIQUE INDEX) and its declared FKs, applying DROP TABLE to clear state. After the replay, every surviving FK is checked against the surviving uniqueness model; an FK whose target tuple is not unique fails the test. The test fk_consistency_across_agpl_migrations runs it over the whole MIGRATIONS slice, and a companion crate can run the same check over the concatenation of core and its own migrations to catch FKs that cross the boundary.

​

Structured store: change tracking and optimistic concurrency

The Store trait is generic CRUD over JSON objects keyed by (tenant_id, account_id, collection, id), plus the JMAP machinery layered on top. get_objects returns (id, Option<value>) for each requested id so an unknown id is reported, never silently dropped. Deletes are soft: an object moves to a deleted_items table and can be restored, which is what backs native undelete and the CalDAV/CardDAV sync-collection recovery of destroyed UIDs. Change tracking implements RFC 8620 §5.2: record_change / record_changes append to a change_log and return the new State. Optimistic concurrency (ADR-010) is built in: record_changes and atomic_set take an optional ifInState, check the current state and write in the same transaction, and return a StateMismatch error if the state moved — no pessimistic locks, no deadlock risk in batched requests. begin_write_transaction plus record_changes_in_txn let a /set handler do all its mutations and the change recording inside one exclusive transaction.

​

Blob store: content-addressed and atomic

BlobStore is binary storage addressed by SHA-256 of the cleartext (ADR-014, amended by the Tier 3.0.19 encryption layout). The filesystem implementation lays blobs out per organization and per account:

• per-account (asymmetric) blobs at {base}/{tenant_id}/{account_id}/{prefix}/{hash},
• tenant-wide (symmetric) blobs at {base}/{tenant_id}/_tenant/{prefix}/{hash}, where _tenant is a reserved sentinel that real account ids may never collide with.

A write is atomic with respect to the storage layer: the file write and the encryption-metadata row insert happen as one operation owned by the store, so callers never coordinate the two halves. On a content match (dedup), the store returns the existing scheme without rewriting the file or regenerating a key. Path components are validated to prevent traversal and to enforce the sentinel rule.

​

Search store: Tantivy with a batching writer

SearchStore is full-text search over documents keyed by (tenant_id, account_id, collection, doc_id) with a single searchable content field. The Tantivy implementation sends index and remove operations to a dedicated background worker over a bounded channel; the worker accumulates mutations and commits in batches (every 500 ms or every 50 documents, whichever comes first). This replaces a per-document commit and removes the writer-lock contention that pattern caused. A flush forces an immediate commit, used by tests and graceful shutdown.

​

Invariants

• Never fail silently. Unknown ids come back as None in their slot; a missing mandatory column, a busy timeout, and a constraint violation all surface as typed StoreError variants. Nothing is dropped.
• Tenant isolation in the storage layer. tenant_id is mandatory on every method; every query filters on it. RBAC is defense in depth on top, not the only line.
• No Mutex<Connection>. Concurrency is the r2d2 pool plus WAL, never a single mutex-guarded connection.
• Migrations are all-or-nothing. One exclusive transaction, fail-loud on error, no manual steps.

​

Configuration

oximail-store does not read configuration files; the binary opens the backend with the values it resolves. The relevant operator knobs are the database path, the optional SQLCipher encryption key (passed to SqliteBackend::open), the connection-pool size (default 8), and the blob-store and search-index base paths. These are documented in the configuration reference. The encryption model behind the blob store (the LUKS-style at-rest scheme, not end-to-end) is covered in encryption at rest.

​

Protocol / JMAP surface

None. oximail-store exposes no JMAP methods and serves no HTTP routes. It is a library consumed by the domain crates: it provides the SqlBackend, Store, BlobStore, and SearchStore traits and their default implementations, plus the Id / BlobId / State primitives that oximail-core re-exports. The JMAP surface that sits on top of this storage lives in the domain crates (oximail-core and the mail, calendar, contact, task, file, and sharing crates).

​

Design decisions

• ADR-002 (SQLite as the default backend) — the community build runs on a single embedded SQLite database with an r2d2 pool; no external database server is required.
• ADR-003 (Tantivy for search) — full-text search is an embedded Tantivy index, not an external search cluster.
• ADR-013 (versioned, auto-applied migrations) — the numbered MIGRATIONS set is applied at boot in one exclusive transaction, fail-loud, with no manual upgrade procedure.
• ADR-015 (tenant isolation in the storage layer) — tenant_id is a mandatory parameter on every store method and every query filters on it, as defense in depth beneath RBAC.
• ADR-010 (optimistic concurrency) — ifInState checks and writes happen in the same transaction; a moved state returns StateMismatch. No pessimistic locks.
• ADR-014 (blob dedup by content hash) and ADR-009 (store raw plus structured) — blobs are addressed by SHA-256 of content; email is parsed once at ingestion and stored as both raw bytes and structured metadata.
• ADR-020 / Tier 3.0.19 (encryption at rest) — the per-account and tenant-wide blob path geometry and the atomic file-plus-metadata write.

ADRs live in the OxiMail server repository; see the ADR index.

​

Standards

• RFC 8620 — JMAP Core. The change-tracking and state model implemented by the Store trait (§5.2 changes, the State string, optimistic concurrency via ifInState).

The rest of the crate is an implementation substrate (SQLite, r2d2, Tantivy, the filesystem) rather than a protocol surface, so it has no additional RFC obligations of its own.

​

Source map & tests

The fk_consistency_across_agpl_migrations test runs the FK validator over the whole MIGRATIONS slice, so adding a migration with an inconsistent foreign key fails the suite before it can reach a database. Every new migration must pass cargo test -p oximail-store.

​

Status

Implemented and in production at v0.30.0: the SqlBackend abstraction over a pooled SQLite database, the Store / BlobStore / SearchStore traits, the 136-migration auto-applied schema with the FK validator, the content-addressed filesystem blob store, and the Tantivy full-text index. This is the storage core the whole server runs on. The community (AGPL) crate ships exactly one SQL backend — SQLite — plus Tantivy and the filesystem blob store. PostgreSQL, object storage, and external search clusters are not part of this crate; the only PostgreSQL references in the source are forward-compatibility comments, not code.