matrix-axon

Axon — Contributor Orientation

Axon is a self-hosted personal agent for Matrix: a persistent state layer (sync, E2EE decryption, search, media proxy) that sits between a user’s homeserver(s) and their clients. Arbitrary clients consume it through a stable, versioned HTTP + WebSocket API at /v1/. See docs/mvp/prd.md for the full product description.

Docs

File Contents
docs/mvp/prd.md Product requirements — what we’re building and why
docs/mvp/tech-spec.md Architecture decisions and tradeoffs
docs/mvp/implementation.md Milestone-by-milestone build plan (authoritative for agentic contributors)
docs/adr/ Architecture decision records — decisions made during implementation
docs/integration-testing.md Running axon against a local Synapse (sync + M3c re-decryption) by hand
docs/README.md Index of every guide under docs/, grouped by audience — add new docs here
docs/install.md End-user install steps per platform; README.md links here for anything beyond the one-liners
docs/building.md Building and running from source (prerequisites, first-time setup, run.sh)
docs/server-admin.md Server operation: sign-in, tokens, SSO, push, bootstrap, management API, TLS
docs/technical-reference.md Architecture, database deadlines, environment variables and config
docs/client-parity.md Human-maintained cross-silo matrix of which /v1/ capabilities each client (TUI, web, future iOS) actually exposes — update it in the same PR that changes a tracked row’s status, from any silo
docs/demo-coverage.md Human-maintained record of which demo scene shows each visually significant capability (ADR 0086) — update it in the same PR that changes what a client renders, from any silo
scripts/demo-stack.sh Brings the ADR 0086 demo world up (up/record/down) for recording client videos. record --capture has ffmpeg record the TUI’s own window (X11/XWayland) instead of a human running a screen recorder. Not a test and not a gate — the stack it starts is meant to stay up while a human (or --capture) records against it.
scripts/demo-web.sh One-shot pipeline for the ADR 0086 web recordings: up/record/assemble/upload/down, or all end-to-end. assemble stitches each platform’s per-scene clips into the single ordered tour demo.html and api-docs.yml expect — see clients/web/README.md § Demo recording.
scripts/demo-all.sh Orchestrates the two scripts above under one command: tui/web run either demo on demand, all [--upload] records (and optionally publishes) both in one pass. Reimplements neither — TUI capture still needs a real X11/XWayland desktop session, so all fails loudly rather than skipping it silently; --web-only opts out on purpose.
scripts/integration-test.sh One-command end-to-end re-decryption test: seeds an encrypted room + key backup via axon-itest, runs axon as a fresh device, and asserts UTDs back-fill. Runnable in CI on demand via .github/workflows/integration.yml (manual workflow_dispatch).
scripts/matrix-oauth-test.sh ADR 0097 black-box and real interoperability lanes against pinned Synapse + MAS: API boundaries, acquisition, fresh SDK-device grant, and unsupported capability. Runs on demand via integration.yml or smoke.yml (notifies the dev room), and after main pushes/nightly via smoke.yml; secret-bearing runs upload no artifacts.

Directory layout

matrix-axon/
  Cargo.toml                 # workspace
  AGENTS.md                  # this file
  CLAUDE.md                  # one-line pointer to this file
  crates/
    axon-server/             # binary; wires components together
    axon-core/               # shared types, errors, config
    axon-store/              # Postgres + sqlx; event store, account data
    axon-sync/               # matrix-rust-sdk sync engine wrapper
    axon-crypto/             # RESERVED STUB — verification lives in axon-sync (ADR 0027)
    axon-search/             # Tantivy index
    axon-media/              # media proxy + disk-cache backend
    axon-api/                # axum HTTP + WS handlers, OpenAPI (utoipa)
    axon-itest/              # dev-only: integration-test seeder (the `seed` binary)
    axon-test-support/       # dev-only: in-memory signing fixtures shared by server tests
  clients/
    tui/                     # axon-tui — terminal client for the Axon API, should grow to support all API endpoints as they are enabled
    web/                     # axon-web (Vite + Preact + TS, ADR 0046) — alpha client
  smoke/                     # black-box smoke harnesses (depend on no axon-* crate; ADR 0025)
    tui/                     # axon-smoke-tui — PTY-drives the real axon-tui against an in-process API stub
                             # also ships axon-demo-tui — the ADR 0086 demo pilot (not a test, not a gate)
    server/                  # axon-smoke-server — black-box API/WS smoke against a real stack
    matrix-oauth/            # ADR 0097 black-box + real Synapse/MAS/SDK QR interoperability
    local-stack/             # axon-smoke-local-stack — boots Synapse + Postgres + axon; writes a JSON manifest
      corpus/                # declarative demo content (ADR 0086); rendered by `up --corpus`
        demo.toml            # personas, spaces, rooms, messages — relative timestamps
        media/               # CC0 avatars (committed) + photos (generated placeholders, not committed)
  openapi/                   # OpenAPI 3.1 spec (source of truth)
  docs/
    mvp/                     # PRD, tech spec, implementation spec (frozen at MVP ship)
    adr/                     # architecture decision records
    README.md                # index of the guides below and beside it
    install.md building.md server-admin.md technical-reference.md   # user and operator guides
    self-hosting.md          # produced in Milestone 13 (deployment docs; still pending)
  docker-compose.yml         # Postgres 16 for dev; Synapse under `integration` profile
  packaging/                 # nFPM .deb/.rpm, and the Homebrew formula and cask templates
  scripts/
    integration-test.sh      # end-to-end E2EE re-decryption test vs local Synapse
  .github/workflows/         # public repo: GitHub-hosted runners have free minutes, so most workflows trigger on push or release (via new tag); a few stay manual-dispatch only for other reasons (expensive/manual runs)
    api-docs.yml             # build the Pages site: homepage + API reference at https://matrix-axon.github.io/matrix-axon/api.html
    check-environment.yml    # local-runner environment check
    cross-build.yml          # fmt, clippy, test, and build for MacOS, Linux, and Windows; on tags, publishes the Homebrew formulas and cask; every server binary carries the web client (ADR 0116); Windows executables are Authenticode-signed when the `ARTIFACT_SIGNING_*` repo variables exist (ADR 0119)
    lint-and-clippy.yml      # required PR/main Rust gate: cargo fmt + clippy + test
    lint-and-test.yml        # cargo fmt + clippy + test
    package.yml              # nFPM .deb/.rpm (amd64 on ubuntu-22.04, arm64 on ubuntu-22.04-arm); tag uploads to GitHub Releases; the binary carries the web client (ADR 0116)
    embed-web.yml            # builds the web client into axon-server, runs the Postgres-gated router tests, then boots the binary and fetches the bundle back (ADR 0116)
    release-plz.yml          # on main (push, or manual dispatch): opens the release PR that bumps the Cargo version, and tags its merge (ADR 0106, release-plz.toml)
    release-notes.yml        # draft release notes grouped by PR label while the release PR is open; publishes them on the tag and opens the CHANGELOG.md PR (ADR 0108, .github/release.yml)
    integration.yml          # selectable E2EE re-decryption and Matrix OAuth QR real-service suites
    matrix-oauth.yml         # reusable Matrix OAuth QR real-service workflow used by integration and smoke
    smoke.yml                # S1 black-box smoke (PR 1: TUI PTY suite; PR 2 added the server gate)
    postgres-tests.yml       # nightly + on demand, never on PRs: every `#[ignore]`d Postgres-backed test via scripts/postgres-tests.sh (one fresh database per test target)
  .github/actions/           # actions relied on by the workflows
    check-environment/       # check environment on local runners before proceeding
    web-bundle/              # build clients/web and export AXON_WEB_DIST, for any job that builds axon-server with `embed-web`

Crates

Each crate’s own Cargo.toml description is the source of truth; this table is the orientation copy.

Crate Purpose
axon-core Shared types, errors, and configuration.
axon-store Postgres-backed event store, room state, and account data.
axon-sync matrix-rust-sdk sync-engine wrapper (Simplified Sliding Sync); owns verification and all crypto-adjacent logic.
axon-crypto Reserved stub — stays empty; verification lives in axon-sync (ADR 0027; see the “axon-crypto stays a stub” convention below).
axon-search Tantivy full-text search index, populated on event ingestion.
axon-media Media proxy with a bounded on-disk LRU cache for mxc:// URLs.
axon-api axum HTTP + WebSocket handlers; OpenAPI spec via utoipa.
axon-server The axon binary — wires the crates together and owns the process.
axon-itest Dev-only end-to-end integration-test seeder (plays a verified Matrix client).
axon-test-support Dev-only in-memory signing fixtures shared by Axon server tests.

Key conventions

Full conventions are in docs/mvp/implementation.md under “Conventions.”

Bootstrap and configuration

axon init (M13, ADR 0051) is the first-run bootstrap — it owns starter-config generation, secret generation, and local-service setup. Config precedence and discovery are described in the Milestone 2 notes below (figment: defaults < TOML < DATABASE_URL < AXON_-prefixed env).

Testing and documentation

Design guardrails

Recurring failure modes distilled from M1–M13 review. They apply across crates and clients.

  1. Encode invariants in types, not comments. Shared mutable state guarded only by a doc-comment is a latent bug — make the type system enforce it (the 7a per-identity lock and run-scoped verification tokens are the model, ADR 0022 / 0027).
  2. Store the key next to shared-but-partitioned state. Whenever there is one physical slot with N logical owners, keep the owner’s key beside the value — or flush/commit the slot on every key change. A single global slot keyed by “current context” silently drops concurrent work when the context switches mid-operation (the PR 192 draft-compose case: a per-room compose buffer and its pending queue must carry the room key or flush on room switch, never live in one “current room” slot).
  3. Bracket every borrow of shared state. A borrow should be save-on-enter / restore-on-exit — ideally structurally via a scope guard, not by remembering to add restore calls at each exit path.
  4. Never infer intent from an incidental value shape. When multiple code paths can produce the same value, that value can’t carry semantics on its own — use explicit sentinels (the search_outbox account-purge sentinel, 9a), not shape-guessing.
  5. Reconciliation must handle absence, not just presence. Prune-on-reconcile is half the job of any lossy / eventually-consistent sync; a reconcile that only adds and never removes is incomplete (the deferred config-drop half of #24, ADR 0024).
  6. In a single-threaded loop, fan out independent I/O. Don’t serialize independent remote calls behind one another when they could run concurrently.
  7. User-entered text must survive a failed mutation. Any operation built from typed input (send, edit, draft) must leave that input recoverable when the call fails — a retryable local echo, a restored composer, a preserved buffer — never error-banner-and-gone. (The web client’s sends have retryable echoes but a failed edit silently discarded the user’s text — WCR-10 in docs/reviews/2026-07-web-client-review.md.) The one exception stays: secret-bearing inputs are consumed once by design, per the secrets rule above.
  8. Every view of server state declares its freshness story. A client store or panel that displays server data must answer, at design time: which live frames update it, what happens on reconnect, or why staleness is acceptable. “Fetched once on mount” is a decision to document, not a default to fall into — the client-side counterpart of guardrail 5’s “reconciliation must handle absence”. (From WCR-06/08: the web thread panel and room list each silently opted out of ADR 0061’s subscribe + gap-fill pattern.)
  9. A predicate’s tests do not cover the code that calls it. Correctness that lives in a pure helper — a visibility filter, a status classifier, an error mapper — attracts exhaustive unit tests, while the branch consuming it typically gets none. So a call site that asks a narrower question than the one it needs, or ignores the answer entirely, ships green with a fully-tested predicate sitting right beside it. Four instances in one review round (#134/#135/#165), across three silos: unread_suppression_reason was handed successor_room().is_some() where Store::list_rooms hides on any tombstone; the branch reading probe_proves_room_absent folded “we could not tell” into “reachable” while the predicate’s own tests asserted !probe_proves_room_absent(Some(500)); the web read effect never called isVisibleTimelineEvent; the TUI room-load path filters its receipt pick through should_show_event and its marker pick, one line below, not at all (#167). When extracting a predicate, build the seam that lets its consumer be tested too — a fake, a loopback server, a pure function taking the outcome — and prefer a type that makes the third state unrepresentable over a bool the call site can quietly collapse (guardrail 4, one level up).

When implementing a new feature:

Instructions after making changes

cargo fmt --all -- --check
cargo clippy --all-features --all-targets -- -D warnings
cargo test --all

The tui smoke lane runs in CI on every pull request (smoke.yml): it needs no Docker and takes about 90 seconds. The Docker-backed lanes stay out of the PR path — they need a true-local Synapse stack — and run on push to main and nightly instead. Use scripts/smoke-gate.sh <mode> before pushing component changes that cross process/network boundaries, so a Docker-lane break is caught before it lands rather than after:

Changed area Smoke gate
smoke/tui or terminal rendering in clients/tui scripts/smoke-gate.sh tui
TUI changes that depend on real Axon/local-stack behavior scripts/smoke-gate.sh tui-true-local
smoke/server, crates/axon-server, crates/axon-api, crates/axon-sync, or crates/axon-store API/sync behavior scripts/smoke-gate.sh server
smoke/local-stack or shared smoke harness behavior scripts/smoke-gate.sh all

The pre-push hook does not run Docker-backed smoke by default. Opt in with RUN_SMOKE=tui, RUN_SMOKE=server, RUN_SMOKE=tui-true-local, or RUN_SMOKE=all — the smoke-gate hook is a no-op unless that variable names a lane, and works the same under git push and jj push.

CI can also run the Docker-backed lanes by hand: dispatch smoke.yml with a lane of tui-true-local, server or all, or with matrix-oauth-all (or one of matrix-oauth-api, -acquire, -grant, -unsupported) to run the Matrix OAuth QR suite alone and notify the axon-developer room; integration.yml offers the same suites without the notification.

Current state

Through M19 landed; MVP has not shipped. MVP is gated on finishing M13 — docs/self-hosting.md and the non-Docker deployment recipes (#221).

What each milestone delivered, and the non-obvious choices made along the way, lives in docs/mvp/implementation.md under that milestone’s heading, with the reasoning in its ADR. It is deliberately not repeated here: this file is the conventions those choices produced, not the record of making them.

This site is open source. Improve this page.