Ekklesia/docs/architecture/decisions/0044-self-hosted-reference-deployment.md

# 0044: Ekklesia ships one self-hosted reference deployment, composed from pinned sources

- Status: Proposed
- Date: 2026-10-01
- Supersedes: none
- Related: [ADR 0003](0003-development-infrastructure.md),
  [ADR 0004](0004-replace-unmaintained-minio.md),
  [ADR 0017](0017-public-request-perimeter-hardening.md),
  [ADR 0018](0018-identity-session-resolution-and-database-transport.md),
  [ADR 0021](0021-modules-as-hostable-products.md),
  [ADR 0030](0030-host-provided-module-object-storage.md),
  [ADR 0031](0031-direct-object-delivery.md),
  [ADR 0032](0032-native-module-realtime-transport.md),
  [ADR 0034](0034-host-owned-browser-authentication.md),
  [ADR 0035](0035-initial-platform-owner-bootstrap.md),
  [ADR 0043](0043-display-offline-code-fallback.md)

## Context

Phases 1–8 built and verified the platform contracts, but `README.md` and
`docs/development/security.md` both record that Ekklesia is not deployed: there is no production
instance, reverse proxy, public listener, Dashboard asset hosting, or deployment automation. The
first real feature module, Signage, now exists in its own repository and needs a production host.

Several gaps only become concrete with a real deployment:

- Production requires certificate-verified PostgreSQL TLS (`verify-full` is the only production
  mode; ADR 0018), including for a database on the same host or Docker network.
- Browsers require a secure context for the display service worker (ADR 0043), and production
  identity rejects `http` and loopback origins. A real, publicly trusted certificate is therefore
  required even on a private network.
- Signage declares a storage bucket, so object storage is no longer optional for this deployment,
  and ADR 0004 left the production provider undecided.
- A module repository is installed by cloning it into `modules/`. It depends on platform packages
  through `workspace:` ranges, so it can only be built inside an Ekklesia workspace, and the
  platform's committed `pnpm-lock.yaml` does not describe it.
- There is no health endpoint, no backup procedure, and no upgrade procedure.

## Decision

Ekklesia provides **one reference self-hosted deployment**, kept in this repository under
`deploy/`, that any organization can run on a single Linux host with Docker Compose. River of
Glory Church is its first instance, but nothing church-specific enters this repository.

### Topology

One host, one Compose project, four long-running services on a private network:

| Service | Role |
| --- | --- |
| `proxy` | Caddy. Terminates TLS, serves the only published ports (443, 80 for redirect), routes `/api/*`, `/display/*`, `/assets/*` and the Dashboard to `api`, and a separate storage origin to `object-storage`. |
| `api` | The `apps/api` image: Core, the enabled modules, and the built Dashboard and display host. Listens on the private network only. |
| `postgres` | PostgreSQL 18 with TLS enabled, using a server certificate issued by an installation-private CA. |
| `object-storage` | SeaweedFS, the same image family as development, reachable by browsers only through the proxy. |

Plus one operator-run, non-restarting service, `migrate`, described below.

### The API serves the Dashboard

The built Dashboard (`surfaces/shell`, `index.html` and its bundle) is served by `apps/api` from
the same origin as `/api`, exactly as the display host already is. This keeps the identity
system's default same-origin shape (`EKKLESIA_DASHBOARD_ORIGIN` unset) and avoids a second static
server. The proxy never serves files from disk.

### TLS everywhere, without weakening ADR 0018

- **Browser-facing:** the proxy obtains publicly trusted certificates through ACME. The reference
  configuration uses the HTTP-01 challenge; the DNS-01 challenge is documented for installations
  that keep Ekklesia off the public Internet. A LAN-only installation can still hold a real
  certificate for an internal hostname this way.
- **Tunnel ingress (optional):** an installation may publish the proxy through an outbound tunnel
  (the reference documents `cloudflared`) instead of opening inbound ports. The tunnel connector
  is a separate Compose service that connects to the proxy over HTTPS and verifies the proxy's
  certificate. The proxy accepts a tunnel-supplied client-address header **only** from the
  connector's network identity. It overwrites the single forwarding header it sends to `api`, and
  `api` trusts only the proxy. An installation that serves local displays should give the same
  hostname a LAN path to the proxy (split-horizon DNS), so displays do not depend on the
  Internet link.
- **Database:** the deployment's setup script creates an installation-private CA once, issues a
  server certificate for the `postgres` service name, and gives the API only the CA certificate
  (`EKKLESIA_DATABASE_CA`). The API connects with `verify-full`. No plaintext or encryption-only
  exception is added to Core.

### Object storage

SeaweedFS is the reference provider for this deployment. Before the first production instance
holds data, the existing storage integration suite (`core/tests/module-storage.integration.ts`,
already run by `pnpm test:migrations:integration`) must pass against the exact SeaweedFS image the
deployment pins. Browsers reach signed object URLs on a dedicated storage origin
(`S3_DELIVERY_ENDPOINT`); the proxy forwards the original `Host` header unchanged so S3 signatures
remain valid (ADR 0031). Any other S3-compatible provider remains a supported substitution.

### Composition from pinned sources

A running installation is defined by an **instance composition**: the platform version and the
exact module sources it includes. The image build:

1. checks out the platform at a release tag;
2. checks out each listed module at a pinned tag or commit SHA into `modules/<id>`;
3. installs with `pnpm install --frozen-lockfile` against a lockfile **committed with the
   instance composition**, not the platform's own lockfile;
4. runs `pnpm verify`-equivalent module and migration validation, then builds.

The instance composition (module list, pins, lockfile, non-secret configuration) lives in a
separate repository owned by the installation. The reference deployment documents its shape and
provides an empty example. Two builds from the same composition produce the same installed code.

This is a deployment mechanism, not a module distribution format. A packaged, installable module
artifact (built outside an Ekklesia workspace, against published SDK versions) is a separate
future decision.

### Migrations stay explicit

The `migrate` service runs `migrations:plan` then, only when an operator invokes it,
`migrations:apply`, from the same image as `api`. The `api` service never applies migrations. The
documented upgrade procedure is: build image → back up → run `migrate` → restart `api`.

### Health

`apps/api` gains two unauthenticated endpoints that reveal nothing about installed modules:

- `GET /api/v1/health/live` — the process is serving HTTP.
- `GET /api/v1/health/ready` — the database is reachable and module startup completed.

Both return a fixed JSON body and `Cache-Control: no-store`. Compose health checks and external
monitoring use them. `pnpm doctor` remains the detailed operator report.

### Backups

The reference deployment includes a scheduled backup job producing a `pg_dump` custom-format
archive and a copy of every bucket under the installation's `S3_BUCKET_PREFIX`, written to a
location outside the host. A restore procedure is documented and must be exercised once before
an installation is treated as in production.

### Secrets

Secrets are supplied to Compose from a root-owned environment file on the host, never committed.
This includes `EKKLESIA_IDENTITY_SECRET`, database and storage credentials, SMTP credentials, and
`EKKLESIA_MODULE_SECRET__*` values. A secret manager is not required.

## Consequences

- Ekklesia has a reproducible, documented way to run in production for the first time, and the
  ten-item launch checklist in `docs/development/security.md` becomes checkable against concrete
  configuration.
- No Core guarantee is relaxed. The database TLS rule, explicit migrations, and the trusted-code
  module model are unchanged.
- The platform and modules begin release tagging, because an instance composition can only pin
  what has a stable name.
- An installation that wants Cloudflare Tunnel, a different proxy, or a managed database can
  substitute it, but the reference configuration is the one CI exercises.
- Building modules inside a platform workspace remains a known limitation until a module
  packaging decision replaces the per-instance lockfile.

## Alternatives

- **Bare-metal systemd services.** Rejected as the reference: harder to reproduce and pin. Remains
  possible for installations that prefer it.
- **Caddy-internal CA for browsers.** Rejected: every display device and admin browser would need
  the CA installed, which is unrealistic for volunteer-run screens.
- **Relaxing database TLS inside a private Docker network.** Rejected: ADR 0018 deliberately has no
  such exception.
- **Applying migrations automatically at API startup.** Rejected: contradicts the explicit-operator
  rule.
- **Using `pnpm install` without a frozen lockfile when adding modules.** Rejected: the installed
  dependency graph would differ between builds.
- **Kubernetes.** Rejected for the reference: disproportionate for a single church server.

## Deferred

- Module packaging and distribution (published SDK packages, signed module artifacts, an
  installer).
- Multi-host, high availability, and managed-database guidance.
- Container image publishing and automated deployment from CI.
- Remote-access gates in front of the Dashboard (for example Cloudflare Access), documented as
  installation choices rather than platform requirements.
- An authorized offline display descriptor or lease (tracked separately; required before relying
  on displays to cold-start without the server).