Ekklesia/deploy

Deployment

This directory holds Ekklesia's reference self-hosted deployment (ADR 0044): one Linux host, one Docker Compose project, and the production container image it runs. Nothing here is specific to one organization.

File	What it is
compose.yaml	The base: api, postgres, object-storage, and the operator-only migrate. Publishes no port.
compose.proxy.yaml	Overlay: Caddy as the TLS proxy, the only service with published ports.
compose.tunnel.yaml	Overlay on top of the proxy: a Cloudflare Tunnel connector.
Caddyfile.example	The proxy's configuration for the default same-origin topology; copy to Caddyfile.
Caddyfile.split.example	The proxy's configuration when the Dashboard and the API have separate hostnames (Split Dashboard and API origins); copy to Caddyfile instead.
postgres/pg_hba.conf	PostgreSQL client authentication: TLS only, SCRAM passwords.
env/	Commented examples of every env file. No values.
scripts/init-tls.sh	Creates the installation-private CA and the postgres and object-storage server certificates.
scripts/gen-secrets.sh	Prints freshly generated secrets to paste into the env files. Writes nothing.
Dockerfile	The one production image, for the API and every operator command (The production image).

The reference deployment

Shape

cd deploy
# Base and proxy: the reference.
docker compose -f compose.yaml -f compose.proxy.yaml up -d
# Base, proxy, and Cloudflare Tunnel.
docker compose -f compose.yaml -f compose.proxy.yaml -f compose.tunnel.yaml up -d
# Base only, behind a reverse proxy the installation already runs.
docker compose -f compose.yaml -f compose.myproxy.yaml up -d

Pass the same -f files to every later up, down, ps, and logs. migrate commands need only the base: docker compose run --rm migrate … from this directory uses compose.yaml by default.

Network	Kind	Members
data	internal	postgres, api, migrate
storage	internal	object-storage, api, migrate
egress	bridge	api, migrate (outbound only, for example an SMTP relay)
edge	bridge, fixed subnet (proxy overlay)	proxy (fixed address), api, object-storage, tunnel (fixed address)

PostgreSQL is only ever on data, so no proxy and no tunnel can reach it. internal networks have no route off the host. Object storage joins edge only when the proxy overlay attaches it.

TLS is everywhere, without any exception in Core:

• Browsers to the proxy: publicly trusted ACME certificates.
• API to PostgreSQL: verify-full against the installation-private CA, passed as PEM text in EKKLESIA_DATABASE_CA. PostgreSQL accepts only TLS 1.3 network connections.
• API (and proxy) to object storage: HTTPS on port 8443 with a certificate from the same CA. Production storage must be https (ADR 0030). Core's S3 client uses Node's default trust store, so Compose adds the CA with NODE_EXTRA_CA_CERTS.

Where configuration and secrets live

File (on the host)	Holds	Read by
deploy/.env (from env/compose.env.example)	Non-secret settings: the image, hostnames, paths, the edge subnet	Compose itself
/etc/ekklesia/ekklesia.env	Everything the API needs, including DATABASE_URL and the S3 keys	api, migrate
/etc/ekklesia/postgres.env	POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB	postgres
/etc/ekklesia/object-storage.env	The S3 key pair	object-storage
/etc/ekklesia/tunnel.env	TUNNEL_TOKEN	tunnel
/etc/ekklesia/tls/	The CA and server certificates and keys	mounted file by file

The secret files are owned by root with mode 600, one per service, so no container receives another service's credentials. None is committed, and deploy/.env is ignored by Git. EKKLESIA_ENV_DIR and EKKLESIA_TLS_DIR in deploy/.env move the directories.

First boot

1. Get the image. Build it (Building) or pull it, and set EKKLESIA_IMAGE in deploy/.env.

2. Create the TLS material, as root:

   sudo sh deploy/scripts/init-tls.sh /etc/ekklesia/tls
   

   It creates the CA once and a certificate for each of postgres and object-storage, gives each key to its server's uid (PostgreSQL 70, SeaweedFS 1000) with mode 600, and prints the exact EKKLESIA_DATABASE_CA="…" line. It never overwrites an existing key or certificate and never prints a private key. ca.key is needed only to issue certificates: keep a copy offline.

3. Generate secrets with sh deploy/scripts/gen-secrets.sh, and paste each line into the file named above it.

4. Write the env files from env/, then install each one owned by root with mode 600:

   sudo install -d -m 755 /etc/ekklesia
   sudo install -o root -g root -m 600 ekklesia.env /etc/ekklesia/ekklesia.env
   

   EKKLESIA_IDENTITY_BASE_URL and EKKLESIA_IDENTITY_TRUSTED_ORIGINS are https:// and your Ekklesia hostname. S3_DELIVERY_ENDPOINT is https:// and your storage hostname.

5. Configure the proxy: copy Caddyfile.example to Caddyfile, and set EKKLESIA_HOSTNAME, EKKLESIA_STORAGE_HOSTNAME, and EKKLESIA_ACME_EMAIL in deploy/.env. Both hostnames must resolve to this host with ports 80 and 443 reachable for HTTP-01, or see DNS-01.

6. Start the data services, then migrate explicitly:

   docker compose -f compose.yaml -f compose.proxy.yaml up -d --wait postgres object-storage
   docker compose run --rm migrate                                   # plan: read-only
   docker compose run --rm migrate node core/dist/cli/apply-migrations.js
   

   Nothing ever applies migrations automatically: the API never does, and migrate runs only when you run it.

7. Start everything: docker compose -f compose.yaml -f compose.proxy.yaml up -d --wait.

8. Check it: curl https://<hostname>/api/v1/health/ready answers {"status":"ready"}, and docker compose run --rm migrate node core/dist/cli/doctor.js reports READY (with warnings for anything optional you left unconfigured, such as email).

The initial owner

docker compose run --rm -it migrate node core/dist/cli/create-initial-owner.js --email [email protected]

It prompts for the password with hidden input and never accepts it on the command line. The owner is created already verified, so they can sign in before any email relay exists (ADR 0035). To supply the password from a secret manager instead, pipe it on standard input and use -T instead of -it.

Upgrading

1. Pull or build the new image, and change EKKLESIA_IMAGE in deploy/.env. Keep the old tag.
2. Back up (Backups).
3. Plan, review, then apply: docker compose run --rm migrate with the new image, then docker compose run --rm migrate node core/dist/cli/apply-migrations.js.
4. Restart on the new image: docker compose -f compose.yaml -f compose.proxy.yaml up -d --wait.

Rolling back

Set EKKLESIA_IMAGE back to the previous tag and run up -d --wait again. That is enough only when the upgrade applied no migrations (the plan in step 3 showed none pending). Migrations are forward-only. If any were applied, restore the backup taken in step 2 before starting the previous image, because it was not built for the newer schema.

Backups

Slice 6 adds the scheduled backup job and the restore procedure (ADR 0044, Backups). Until then, take a pg_dump and a copy of every bucket before each upgrade by hand.

Split Dashboard and API origins

By default one hostname serves the Dashboard (/), the API (/api/*), and displays (/display/*). An installation may instead give the Dashboard its own hostname, for example dash.example.org beside api.example.org (ADR 0045). The same image serves both; only configuration differs.

Hostname	Serves	Refuses (404)
Dashboard (EKKLESIA_DASHBOARD_HOSTNAME)	/ and /assets/ekklesia-dashboard/*	everything else, including /api/*
API (EKKLESIA_HOSTNAME)	/api/*, /display/*, /assets/ekklesia-display/*	everything else, including /
Storage (EKKLESIA_STORAGE_HOSTNAME)	signed object URLs, unchanged

Displays live on the API's hostname (https://api.example.org/display/<module>), because their device cookies are bound to it and their offline cache keeps only same-origin responses. That hostname is what every screen and paired device depends on: choose it once.

To run split:

1. Copy Caddyfile.split.example to Caddyfile instead of Caddyfile.example.
2. In deploy/.env, set EKKLESIA_HOSTNAME to the API's hostname and EKKLESIA_DASHBOARD_HOSTNAME to the Dashboard's.
3. In ekklesia.env, set EKKLESIA_IDENTITY_BASE_URL to https:// and the API's hostname, EKKLESIA_DASHBOARD_ORIGIN to https:// and the Dashboard's, and EKKLESIA_IDENTITY_TRUSTED_ORIGINS to the Dashboard's origin. The API always trusts its own origin, so it need not be listed.

Both hostnames must belong to one registrable domain (dash.example.org and api.example.org, not dash.example.org and api.example.net). The session cookie stays host-only on the API's hostname; browsers send it from the Dashboard only between origins of the same site. Doctor reports the topology and warns when the two hostnames are not siblings.

The Dashboard learns the API's origin at run time: the API writes its identity base URL into the page it serves (<meta name="ekklesia-api-origin">). Nothing about the topology is built into the image.

Behind an existing reverse proxy

Run the base alone and attach api and object-storage to your proxy's network with an overlay of your own, for example compose.myproxy.yaml:

services:
  api:
    networks:
      proxy: {}
    environment:
      # Exactly one trusted peer: your proxy, at an address that cannot change.
      EKKLESIA_CLIENT_ADDRESS_MODE: trusted-proxy
      EKKLESIA_CLIENT_ADDRESS_HEADER: x-forwarded-for
      EKKLESIA_CLIENT_ADDRESS_TRUSTED_PROXIES: 10.0.100.254/32
  object-storage:
    networks:
      proxy: {}

networks:
  proxy:
    external: true
    name: your-proxy-network

Then make your proxy do what Caddyfile.example does:

• forward /, /api/*, /display/*, and /assets/* on the Ekklesia hostname to api:3000;
• overwrite X-Forwarded-For with the one client address it resolved, and strip CF-Connecting-IP, X-Real-IP, Forwarded, and True-Client-IP;
• stream responses on /api/v1/module-realtime/* without buffering (Server-Sent Events);
• set no request-body limit below 32 MiB; the API enforces its own ceilings;
• forward the storage hostname to https://object-storage:8443, verifying it against /etc/ekklesia/tls/ca.crt with server name object-storage, and pass the original Host through unchanged, or every signed URL fails (ADR 0031).

The proxy's address must be fixed and must never be handed to anything else. With Docker, give its network an explicit subnet, limit ip_range to part of it, and fix the proxy's address outside that range, as compose.proxy.yaml does.

Fixed addresses on edge

compose.proxy.yaml gives edge the subnet 10.249.250.0/24 and lets Docker assign dynamic addresses only from its lower half (10.249.250.0/25). The proxy (.254) and the tunnel connector (.253) take fixed addresses from the top of the upper half, which Docker never assigns to anything else. That is what makes "only the proxy" and "only the connector" expressible as /32 trust entries. If the subnet collides with one on your host, change EKKLESIA_EDGE_SUBNET, EKKLESIA_EDGE_DYNAMIC_RANGE, EKKLESIA_PROXY_ADDRESS, and EKKLESIA_TUNNEL_ADDRESS together, keeping the fixed addresses at the top and outside the dynamic range.

Cloudflare Tunnel

compose.tunnel.yaml runs cloudflared with a token, so the tunnel is managed in the Cloudflare dashboard. No Cloudflare API is called from here.

1. In Cloudflare Zero Trust, under Networks → Tunnels, create a Cloudflared tunnel. Copy its token into /etc/ekklesia/tunnel.env as TUNNEL_TOKEN.
2. Add a public hostname (shown as a published application route in newer dashboards) for the Ekklesia hostname. Service type HTTPS, URL proxy:443.
3. Under that hostname's TLS settings, set Origin Server Name to the same Ekklesia hostname, and leave No TLS Verify off. The connector then verifies the proxy's certificate for that name.
4. Add a public hostname for the storage hostname, the same way, with its own origin server name. In the split topology, add one for the Dashboard's hostname too: three in all, each with its own origin server name.
5. Start with all three files. The proxy honors CF-Connecting-IP only from the connector's fixed address. From any other peer it is stripped.

The proxy's certificate must still be publicly trusted, and an HTTP-01 challenge cannot reach the proxy through a tunnel, because Cloudflare terminates TLS at its edge. A tunnel-only installation therefore uses DNS-01. An installation that also serves local displays should give the same hostnames a LAN path to the proxy (split-horizon DNS), so displays do not depend on the Internet link.

DNS-01

For an installation off the public Internet, LAN-only, or tunnel-only. DNS-01 needs a DNS provider module, which the stock Caddy image does not contain:

1. Build a Caddy image with the module (for example xcaddy build --with github.com/caddy-dns/cloudflare) and pin it by digest.
2. Override the proxy's image: in an overlay of your own. Give it CLOUDFLARE_API_TOKEN (a token scoped to DNS edits for the zone) through a root-owned env file.
3. Enable the commented acme_dns line in your Caddyfile.

Launch checklist

The ten items in Security § launch checklist mapped to this deployment:

#	Item	Where it is met
1	The proxy forwards the Dashboard and /api on the intended origin	Caddyfile.example routes /, /api/*, /display/*, /assets/* on one hostname to api; the API serves the Dashboard itself. In the split topology, Caddyfile.split.example gives each hostname exactly its own paths and the exact trusted-origin CORS of ADR 0045 covers the rest.
2	The origin port is unreachable from the Internet	The base publishes no port; only proxy publishes 80 and 443.
3	The origin port is unreachable from untrusted networks	api is reachable only on Docker networks; EKKLESIA_PUBLISH_ADDRESS limits the proxy to one interface. Firewall the host itself.
4	The proxy overwrites the trusted client-IP header	header_up X-Forwarded-For {client_ip} in Caddyfile.example.
5	Rotating forwarding headers cannot rotate the rate-limit bucket	The same, plus stripping CF-Connecting-IP, X-Real-IP, Forwarded, and True-Client-IP; the API overwrites its internal header. Proven by the smoke test.
6	The proxy peer is the only trusted proxy	compose.proxy.yaml sets EKKLESIA_CLIENT_ADDRESS_TRUSTED_PROXIES to the proxy's fixed /32.
7	Proxy logs record no cookies, credentials, bodies, or forwarding chains	Caddy access logs are off in Caddyfile.example. If you enable them, exclude request headers.
8	PostgreSQL only from the API, over verified TLS	postgres is only on internal data; pg_hba.conf rejects non-TLS; verify-full with EKKLESIA_DATABASE_CA. Proven by the smoke test.
9	The deployed login and logout flow keeps its contract	Unchanged by the proxy; the smoke test signs in and out through it.
10	A supported way to create the first production identity	create-initial-owner.js through migrate (The initial owner). Exercised by the smoke test.

Testing this deployment

• pnpm deploy:config validates the base, base + proxy, and base + proxy + tunnel with the example env files. CI runs it in pnpm ci:verify.

• pnpm deploy:smoke (scripts/smoke-deploy.sh) runs the whole deployment end to end, against a platform-only image and a disposable PostgreSQL and SeaweedFS that exist only for that run. It follows the first boot above and checks the following through the proxy:

  • readiness and the Dashboard;
  • a signed object URL fetched through the storage hostname;
  • the client-address chain, both from an ordinary peer and from the tunnel connector's address;
  • initial-owner sign-in and sign-out;
  • that the API's database connection is TLS 1.3, and that it is refused by a foreign CA and by a wrong host name;
  • Doctor.

  EKKLESIA_SMOKE_TOPOLOGY=split runs the split topology instead: an image with the repository's module template installed as a test-only module, every hostname's served and refused paths, exact CORS for the Dashboard's origin and none for others, and then a real headless Chromium (pinned by digest, driven over the DevTools protocol) that signs in on the Dashboard's hostname, checks the session cookie lives only on the API's, mounts the module's UI from the API's origin and sees its own credentialed requests succeed, mounts its display on the API's hostname with its service worker, and signs out.

  CI runs both topologies on every change.

• Test-only: the smoke test runs Caddy with its internal CA (local_certs), because a test run has no public hostname. Never use it in an installation; Caddyfile.example does not contain it.

The smoke test uses no module, so it does not exercise realtime streams or large uploads. Through this same proxy configuration, a scratch module confirmed both:

• a Server-Sent Event crossed the proxy in milliseconds;
• a 32 MiB streamed upload passed with and without Content-Length;
• one byte more was refused by the API's own 413 REQUEST_BODY_TOO_LARGE.

The production image

Dockerfile builds one image for two jobs: the long-running api service, and every operator-run command (migrations, Doctor, initial-owner creation, module enablement). There is no second image and no second build.

What it builds

The build context is an already-assembled workspace: this repository, plus zero or more module packages already present in modules/<id>, plus a pnpm-lock.yaml describing exactly that set. The Dockerfile clones nothing and chooses no modules. It builds what it is given. With no modules/ directory, it builds the platform alone; CI builds that case on every change.

Assembling the context (checking out the platform at a release tag, each module at a pinned tag or commit, and the instance's own lockfile) belongs to the installation's instance composition, not to this file.

Stage	Does
base	node:24.18.0-trixie-slim, pinned by its multi-architecture index digest. Debian, so glibc, because module dependencies such as sharp ship glibc prebuilt binaries. Every stage starts here.
toolchain	Enables pnpm through Corepack, at exactly the root packageManager version.
manifests	Collects the lockfile, workspace configuration, .npmrc, and every workspace package.json at its repository path.
build	pnpm install --frozen-lockfile, then pnpm boundaries, pnpm modules:validate, pnpm migrations:validate, and pnpm build, against exactly the modules in the context. Never uses host node_modules: .dockerignore excludes them.
prod-deps	The same lockfile with --prod: production dependencies only.
runtime	Production dependencies plus built outputs, owned by root and run as the unprivileged node user (uid 1000). No pnpm, no compiler, no curl.

The runtime keeps the repository layout at /opt/ekklesia, because that layout is how the platform finds its files:

• the API takes the repository root as three directories above apps/api/dist/, and reads package.json, packages/module-sdk/package.json, and modules/ beneath it;
• every CLI takes its working directory as the root and also reads Core's migrations in core/drizzle/, so commands must run from /opt/ekklesia, the image's WORKDIR;
• the Dashboard and display host files are read from surfaces/shell/dist/browser/;
• each module directory is copied whole, minus node_modules, because each manifest names its own entrypoints, UI artifacts, and migrations directory.

Building

docker build \
  --file deploy/Dockerfile \
  --build-arg VERSION=0.1.1 \
  --build-arg REVISION="$(git rev-parse HEAD)" \
  --build-arg SOURCE=https://git.example.org/your-org/ekklesia \
  --tag ekklesia:0.1.1 \
  .

The three build arguments become the OCI labels org.opencontainers.image.version, .revision, and .source. They have no defaults: the repository cannot know an installation's Git host. The image also records its base image name and digest. BuildKit is required (Docker's default builder); set DOCKER_BUILDKIT=1 if your Docker CLI has no buildx plugin.

No secret is a build argument, and none enters any layer. .env files are excluded from the build context entirely, and no registry credentials are needed.

A module missing a production dependency fails the build, not the API. The runtime receives production dependencies only, so a module that imports a package at run time but declares it only as a development or peer dependency would build and then fail at startup. pnpm boundaries runs in the build stage and rejects that module first (runtime-imports-are-production-dependencies, see Repository boundaries). The same applies to every other boundary rule: a module the platform's own CI would reject cannot be built into an image.

A platform-only image does not exercise native module dependencies. With no modules in the context, nothing like sharp is installed, so the CI image build proves nothing about them. An instance build that includes such a module must smoke-test the built image itself: start it, confirm /api/v1/health/ready answers 200, and exercise the module's native code path (for example, process one image).

Configuration

Configuration comes only from the container environment, never from a file inside the image. The image sets exactly three variables itself:

Variable	Image value	Why
NODE_ENV	production	Stated rather than relied on. Anything other than development already validates as production.
EKKLESIA_API_HOST	0.0.0.0	Reachable from other containers. Outside a container the platform default stays 127.0.0.1.
EKKLESIA_API_PORT	3000	The platform default, made explicit; EXPOSE 3000.

Everything else is the ordinary host configuration in API § Required configuration, supplied by the reference deployment's ekklesia.env. EKKLESIA_DATABASE_CA is the CA's PEM text itself, on one double-quoted line with \n escapes. Compose expands those escapes; plain docker run --env-file does not, which is why every command here goes through Compose.

Never publish the API's port to the host or the Internet. Only the proxy publishes ports.

Process, signals, and the read-only posture

compose.yaml runs api and migrate with init: true, read_only: true, tmpfs: /tmp, cap_drop: [ALL], and no-new-privileges. That is the supported production posture:

• init: true runs a minimal init as PID 1 that forwards signals and reaps children, so docker stop reaches the API's own SIGTERM handler promptly. The API then drains in-flight requests, stops modules in reverse order, and closes its pool.
• Read-only: the image was verified under exactly these flags. The API reached healthy, and the migration and Doctor commands ran. Nothing wrote to /tmp, which stays mounted as headroom for module code that needs scratch space. The installation, /home/node included, is not writable, so anything that needs a writable path other than /tmp is a defect to report, not to work around.

Health

The image's HEALTHCHECK calls GET /api/v1/health/ready on 127.0.0.1 with Node's built-in fetch, every 30 seconds after a 30-second start period, with three retries. Readiness means the database answered within 2 seconds and the instance is not shutting down; see API § Health. Because it probes 127.0.0.1, keep EKKLESIA_API_HOST at 0.0.0.0 (or any address that includes loopback) inside the container. The API trusts only the proxy's address for client-address headers, so the loopback probe is an ordinary direct peer.

An external monitor can use the same endpoint through the proxy. Use /api/v1/health/live to ask only whether the process is serving.

Operator commands

The same image runs every CLI, directly with node core/dist/cli/<command>.js, through the migrate service. The pnpm scripts of the same names rebuild first and read .env, so they are for a development checkout, not for the image.

migrate has profiles: ["ops"], so up never starts it. It never restarts, has no health check, and its default command is the read-only plan. Nothing applies migrations automatically.

Run operator commands with the same Compose files as the stack. COMPOSE_FILE in deploy/.env (set in env/compose.env.example) does that for every plain docker compose command below. It matters for Doctor in particular: Doctor checks the client-address configuration of the process it runs in. The proxy overlay gives migrate the API's trusted-proxy settings, so Doctor reports what the API runs. Without the overlay, Doctor reports direct mode for an API that trusts its proxy. The running API's own value is shown by docker compose exec api printenv EKKLESIA_CLIENT_ADDRESS_MODE.

Task	Command (from deploy/)
Plan migrations (read-only)	docker compose run --rm migrate
Apply migrations	docker compose run --rm migrate node core/dist/cli/apply-migrations.js
Doctor (read-only report)	docker compose run --rm migrate node core/dist/cli/doctor.js
Create the initial owner (interactive)	docker compose run --rm -it migrate node core/dist/cli/create-initial-owner.js --email [email protected]
Enable or disable a module	docker compose run --rm migrate node core/dist/cli/set-module-enabled.js --enable --module <id> (or --disable)

The plan, apply, Doctor, and initial-owner commands are exercised by the smoke test; module enablement is not.

Size

The platform-only image is about 378 MB: the Debian slim Node base (about 230 MB), production dependencies (about 141 MB), and built outputs (about 6 MB). A rebuild that reuses a warm pnpm store cache can come out about 21 MB larger. When esbuild's install script runs, it hard-links its bin/esbuild to the native binary it ships beside it; when pnpm restores esbuild from the store's cached post-install state instead, that file is a second full copy. Both are esbuild, one of the unused tools below.

About 57 MB of the production dependencies are development tools that no runtime code imports: vitest, vite, drizzle-kit, tsx, esbuild, and their native bindings. They arrive as optional peer dependencies of Better Auth (and esbuild through the module UI builder), which pnpm resolves because the workspace already has them. Excluding them means changing dependency resolution and the lockfile, which is separate work.