Files
ekklesia f9585cc7ec
CI / verify (pull_request) Successful in 1m5s
CI / infrastructure-smoke (pull_request) Successful in 36s
CI / migration-smoke (pull_request) Successful in 1m51s
CI / package-release (pull_request) Successful in 34s
CI / image-build (pull_request) Successful in 59s
CI / deploy-smoke (same-origin) (pull_request) Successful in 47s
CI / deploy-smoke (split) (pull_request) Successful in 2m21s
CI / package-publication (pull_request) Successful in 49s
chore(release): prepare 0.2.4
2026-10-10 11:54:00 -05:00

39 KiB

Deployment

This directory holds Ekklesia's reference self-hosted deployment (ADR 0044): one Linux host, one Docker Compose project, and the production container image it runs. Nothing here is specific to one organization.

File What it is
compose.yaml The base: api, postgres, object-storage, and the operator-only migrate. Publishes no port.
compose.proxy.yaml Overlay: Caddy as the TLS proxy, the only service with published ports.
compose.tunnel.yaml Overlay on top of the proxy: a Cloudflare Tunnel connector.
Caddyfile.example The proxy's configuration for the default same-origin topology; copy to Caddyfile.
Caddyfile.split.example The proxy's configuration when the Dashboard and the API have separate hostnames (Split Dashboard and API origins); copy to Caddyfile instead.
postgres/pg_hba.conf PostgreSQL client authentication: TLS only, SCRAM passwords.
env/ Commented examples of every env file. No values.
scripts/init-tls.sh Creates the installation-private CA and the postgres and object-storage server certificates.
scripts/gen-secrets.sh Prints freshly generated secrets to paste into the env files. Writes nothing.
Dockerfile The one production image, for the API and every operator command (The production image).

The reference deployment

Shape

cd deploy
# Base and proxy: the reference.
docker compose -f compose.yaml -f compose.proxy.yaml up -d
# Base, proxy, and Cloudflare Tunnel.
docker compose -f compose.yaml -f compose.proxy.yaml -f compose.tunnel.yaml up -d
# Base only, behind a reverse proxy the installation already runs.
docker compose -f compose.yaml -f compose.myproxy.yaml up -d

Pass the same -f files to every later up, down, ps, and logs. migrate commands need only the base: docker compose run --rm migrate … from this directory uses compose.yaml by default.

Network Kind Members
data internal postgres, api, migrate
storage internal object-storage, api, migrate
egress bridge api, migrate (outbound only, for example an SMTP relay)
edge bridge, fixed subnet (proxy overlay) proxy (fixed address), api, object-storage, tunnel (fixed address)

PostgreSQL is only ever on data, so no proxy and no tunnel can reach it. internal networks have no route off the host. Object storage joins edge only when the proxy overlay attaches it.

TLS is everywhere, without any exception in Core:

  • Browsers to the proxy: publicly trusted ACME certificates.
  • API to PostgreSQL: verify-full against the installation-private CA, passed as PEM text in EKKLESIA_DATABASE_CA. PostgreSQL accepts only TLS 1.3 network connections.
  • API (and proxy) to object storage: HTTPS on port 8443 with a certificate from the same CA. Production storage must be https (ADR 0030). Core's S3 client uses Node's default trust store, so Compose adds the CA with NODE_EXTRA_CA_CERTS.

Where configuration and secrets live

File (on the host) Holds Read by
deploy/.env (from env/compose.env.example) Non-secret settings: the image, hostnames, paths, the edge subnet Compose itself
/etc/ekklesia/ekklesia.env Everything the API needs, including DATABASE_URL and the S3 keys api, migrate
/etc/ekklesia/postgres.env POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB postgres
/etc/ekklesia/object-storage.env The S3 key pair object-storage
/etc/ekklesia/tunnel.env TUNNEL_TOKEN tunnel
/etc/ekklesia/tls/ The CA and server certificates and keys mounted file by file
/var/lib/ekklesia/modules/ Installed modules (Installed modules); not a secret api, migrate, read-only

The secret files are owned by root with mode 600, one per service, so no container receives another service's credentials. None is committed, and deploy/.env is ignored by Git. EKKLESIA_ENV_DIR and EKKLESIA_TLS_DIR in deploy/.env move the directories.

First boot

  1. Get the image. Build it (Building) or pull it, and set EKKLESIA_IMAGE in deploy/.env.

  2. Create the TLS material, as root:

    sudo sh deploy/scripts/init-tls.sh /etc/ekklesia/tls
    

    It creates the CA once and a certificate for each of postgres and object-storage, gives each key to its server's uid (PostgreSQL 70, SeaweedFS 1000) with mode 600, and prints the exact EKKLESIA_DATABASE_CA="…" line. It never overwrites an existing key or certificate and never prints a private key. ca.key is needed only to issue certificates: keep a copy offline.

  3. Generate secrets with sh deploy/scripts/gen-secrets.sh, and paste each line into the file named above it.

  4. Write the env files from env/, then install each one owned by root with mode 600:

    sudo install -d -m 755 /etc/ekklesia
    sudo install -o root -g root -m 600 ekklesia.env /etc/ekklesia/ekklesia.env
    

    EKKLESIA_IDENTITY_BASE_URL and EKKLESIA_IDENTITY_TRUSTED_ORIGINS are https:// and your Ekklesia hostname. S3_DELIVERY_ENDPOINT is https:// and your storage hostname.

  5. Create module storage, empty, even if you install no module yet:

    sudo install -d -o root -g root -m 755 /var/lib/ekklesia/modules
    

    api and migrate refuse to start without it: Compose never creates it for you, so a mistyped path cannot quietly become an empty directory. EKKLESIA_MODULES_HOST_DIR in deploy/.env moves it.

  6. Configure the proxy: copy Caddyfile.example to Caddyfile, and set EKKLESIA_HOSTNAME, EKKLESIA_STORAGE_HOSTNAME, and EKKLESIA_ACME_EMAIL in deploy/.env. Both hostnames must resolve to this host with ports 80 and 443 reachable for HTTP-01, or see DNS-01.

  7. Start the data services, then migrate explicitly:

    docker compose -f compose.yaml -f compose.proxy.yaml up -d --wait postgres object-storage
    docker compose run --rm migrate                                   # plan: read-only
    docker compose run --rm migrate node core/dist/cli/apply-migrations.js
    

    Nothing ever applies migrations automatically: the API never does, and migrate runs only when you run it.

  8. Start everything: docker compose -f compose.yaml -f compose.proxy.yaml up -d --wait.

  9. Check it: curl https://<hostname>/api/v1/health/ready answers {"status":"ready"}, and docker compose run --rm migrate node core/dist/cli/doctor.js reports READY (with warnings for anything optional you left unconfigured, such as email).

The initial owner

docker compose run --rm -it migrate node core/dist/cli/create-initial-owner.js --email [email protected]

It prompts for the password with hidden input and never accepts it on the command line. The owner is created already verified, so they can sign in before any email relay exists (ADR 0035). To supply the password from a secret manager instead, pipe it on standard input and use -T instead of -it.

Installed modules

The image contains no modules (ADR 0048). Installed modules live on the host, in module storage: /var/lib/ekklesia/modules, or EKKLESIA_MODULES_HOST_DIR. Compose mounts it read-only at /var/lib/ekklesia/modules in api and migrate, which is where EKKLESIA_MODULES_DIR points. Ekklesia never writes there. Installing, updating, or removing a module never needs a new image, only a restart.

What belongs there. One directory per module, each extracted from that module's Ekklesia Module Package (ADR 0049): one archive, <id>-<version>-<target>.tar.gz, and the SHA-256 published beside it. A package holds the module's generated package.json, its manifest, ekklesia.package.json, bundled backend code, built browser artifacts, migrations, and THIRD_PARTY_NOTICES (Module development). Copying a module's dist/ alone is not enough. Files are owned by root and readable by everyone (directories 755, files 644), so the API's unprivileged user can read them and never change them. The digest proves the archive is the one you were given; it does not prove who published it.

Nothing else belongs there. Every non-hidden directory must be a valid module, or the API does not start. That includes lost+found, so if module storage is a file system of its own, mount it one level up and point EKKLESIA_MODULES_HOST_DIR at a directory inside it. Hidden entries are ignored, which is what makes staging possible.

Changes take effect only at a restart. A running API never notices module storage changing, and never loads or unloads a module while it runs. Change module storage only while the API is stopped: it reads browser artifacts from disk as it serves them, so files changed under a running API would serve one version's browser code beside another's server code.

To install or update a module, with <id> its module id and the package and its .sha256 in the current directory, first verify it with the image's own verifier. It checks the archive's digest, its safety (no links, special files, or escaping paths), every file's digest, the package's identity, target, and Node.js range against this platform, runs the platform's module and migration validation, and loads the backend once in a short-lived process with no environment. Only then does it write the verified module directory, to a new directory outside module storage:

The backend probe executes the package's server-side code; the verifier container is its isolation boundary. Keep the general-purpose /tmp mount noexec, and give only this one-shot verifier a separate executable scratch tmpfs. The mount below is ephemeral, bounded at 1100 MiB (just above the package format's 1 GiB expanded limit), nodev, nosuid, and writable only by the image's unprivileged node user. EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR tells the CLI where that mount is; the verifier creates a unique child for each run and removes it on success or failure.

mkdir -m 777 staging        # writable by the image's unprivileged user; nothing else is
docker run --rm --network none --read-only \
  --tmpfs /tmp:rw,nosuid,nodev,noexec,size=64m \
  --tmpfs /run/ekklesia-verifier:rw,nosuid,nodev,exec,size=1100m,mode=0700,uid=1000,gid=1000 \
  --cap-drop ALL --security-opt no-new-privileges \
  -e EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR=/run/ekklesia-verifier \
  -v "$PWD:/package:ro" -v "$PWD/staging:/staging" \
  --entrypoint node "$EKKLESIA_IMAGE" core/dist/cli/verify-module-package.js \
  /package/<id>-<version>-any.tar.gz --sha256-file /package/<id>-<version>-any.tar.gz.sha256 \
  --extract-to /staging/<id>

Then install the verified directory:

docker compose stop api
# Stage under a hidden name, which discovery ignores, then rename it into place in one step.
sudo cp -R staging/<id> /var/lib/ekklesia/modules/.incoming-<id>
sudo chown -R root:root /var/lib/ekklesia/modules/.incoming-<id>
sudo chmod -R a+rX,go-w /var/lib/ekklesia/modules/.incoming-<id>
sudo mv /var/lib/ekklesia/modules/<id> /var/lib/ekklesia/modules/.previous-<id>   # update only
sudo mv /var/lib/ekklesia/modules/.incoming-<id> /var/lib/ekklesia/modules/<id>
docker compose run --rm migrate node core/dist/cli/validate-modules.js
docker compose run --rm migrate                                         # plan: review it
docker compose run --rm migrate node core/dist/cli/apply-migrations.js  # only when the plan is right
docker compose up -d --wait api

Back up before an update that brings migrations: migrations are forward-only. A newly installed module is enabled unless an administrator disabled it before; set its configuration and grants as its documentation says. Keep .previous-<id> until the update has proven itself, then delete it.

To remove a module, stop the API, delete (or move away) its directory, and start the API again. Remove first any module that requires it, or the API refuses to start. Removing a module's code deletes nothing else. Its tables and rows, its buckets and objects, its enablement, configuration, Dashboard placement, and grants all stay. Its applied migration history stays too, and the migration plan and Doctor report it as retained history for a module that is not installed. That is information, not a problem, and it blocks no other migration. To stop a module without removing it, disable it instead (set-module-enabled.js --disable, Operator commands). There is no purge: deleting a module's data is a separate operation that does not exist yet.

To restore a removed module, verify and install its package again as above, at the same version or a newer one. It finds its data, its settings, and its history where it left them. Its history is strict again: a package that no longer contains a migration that was applied, or contains one with different contents, is refused by the migration plan, so an older version cannot be installed over newer data.

Module storage is installation state: back it up with the database and the buckets.

Upgrading

  1. Pull or build the new image, and change EKKLESIA_IMAGE in deploy/.env. Keep the old tag. Installed modules stay where they are, in module storage.
  2. Back up (Backups).
  3. Check the installed modules against the new platform: docker compose run --rm migrate node core/dist/cli/validate-modules.js. A module whose compatibility.platform range excludes the new version fails here, and would stop the API from starting: update it first, or stay on the old image.
  4. Plan, review, then apply: docker compose run --rm migrate with the new image, then docker compose run --rm migrate node core/dist/cli/apply-migrations.js.
  5. Restart on the new image: docker compose -f compose.yaml -f compose.proxy.yaml up -d --wait.

Rolling back

Set EKKLESIA_IMAGE back to the previous tag and run up -d --wait again. That is enough only when the upgrade applied no migrations (the plan in step 4 showed none pending). Migrations are forward-only. If any were applied, restore the backup taken in step 2 before starting the previous image, because it was not built for the newer schema. Module storage is not part of the image, so a rollback does not change installed modules.

Backups

Slice 6 adds the scheduled backup job and the restore procedure (ADR 0044, Backups). Until then, take a pg_dump, a copy of every bucket, and a copy of module storage before each upgrade by hand.

Split Dashboard and API origins

By default one hostname serves the Dashboard (/), the API (/api/*), and displays (/display/*). An installation may instead give the Dashboard its own hostname, for example dash.example.org beside api.example.org (ADR 0045). The same image serves both; only configuration differs.

Hostname Serves Refuses (404)
Dashboard (EKKLESIA_DASHBOARD_HOSTNAME) / and /assets/ekklesia-dashboard/* everything else, including /api/*
API (EKKLESIA_HOSTNAME) /api/*, /display/*, /assets/ekklesia-display/* everything else, including /
Storage (EKKLESIA_STORAGE_HOSTNAME) signed object URLs, unchanged

Displays live on the API's hostname (https://api.example.org/display/<module>), because their device cookies are bound to it and their offline cache keeps only same-origin responses. That hostname is what every screen and paired device depends on: choose it once.

To run split:

  1. Copy Caddyfile.split.example to Caddyfile instead of Caddyfile.example.
  2. In deploy/.env, set EKKLESIA_HOSTNAME to the API's hostname and EKKLESIA_DASHBOARD_HOSTNAME to the Dashboard's.
  3. In ekklesia.env, set EKKLESIA_IDENTITY_BASE_URL to https:// and the API's hostname, EKKLESIA_DASHBOARD_ORIGIN to https:// and the Dashboard's, and EKKLESIA_IDENTITY_TRUSTED_ORIGINS to the Dashboard's origin. The API always trusts its own origin, so it need not be listed.

Both hostnames must belong to one registrable domain (dash.example.org and api.example.org, not dash.example.org and api.example.net). The session cookie stays host-only on the API's hostname; browsers send it from the Dashboard only between origins of the same site. Doctor reports the topology and warns when the two hostnames are not siblings.

The Dashboard learns the API's origin at run time: the API writes its identity base URL into the page it serves (<meta name="ekklesia-api-origin">). Nothing about the topology is built into the image.

Storage delivery: CORS is the deployment's

Browsers fetch objects directly from the storage hostname through signed, expiring URLs (ADR 0031). Which pages may read those bytes with fetch() is decided by the delivery layer, the proxy, and never by the object store. That policy therefore stays the same with SeaweedFS, R2, or any other S3 backend behind it, and an installation that changes backends changes nothing about CORS.

The reference proxy's storage block:

  • grants CORS only to the origins whose pages read objects with fetch(): the Ekklesia origin in the same-origin topology; the API origin (displays) and the Dashboard origin (a module's preview) in the split topology. An <img> needs no CORS and is unaffected;
  • allows GET and HEAD only, and no request header beyond the CORS-safelisted ones, always with Vary: Origin;
  • answers preflight OPTIONS requests itself, so they never reach storage;
  • removes the object store's own CORS headers. SeaweedFS's default reflects any origin and advertises PUT, POST, and DELETE.

The proxy reaches storage over HTTP/1.1 only. Over HTTP/2, SeaweedFS answers a conditional request with a 304 that declares a Content-Length it never sends. The proxy then resets the browser's stream, and every reload of a cached image fails with ERR_HTTP2_PROTOCOL_ERROR. Browsers still reach the proxy itself over HTTP/2 and HTTP/3.

Behind an existing reverse proxy

Run the base alone and attach api and object-storage to your proxy's network with an overlay of your own, for example compose.myproxy.yaml:

services:
  api:
    networks:
      proxy: {}
    environment:
      # Exactly one trusted peer: your proxy, at an address that cannot change.
      EKKLESIA_CLIENT_ADDRESS_MODE: trusted-proxy
      EKKLESIA_CLIENT_ADDRESS_HEADER: x-forwarded-for
      EKKLESIA_CLIENT_ADDRESS_TRUSTED_PROXIES: 10.0.100.254/32
  object-storage:
    networks:
      proxy: {}

networks:
  proxy:
    external: true
    name: your-proxy-network

Then make your proxy do what Caddyfile.example does:

  • forward /, /api/*, /display/*, and /assets/* on the Ekklesia hostname to api:3000;
  • overwrite X-Forwarded-For with the one client address it resolved, and strip CF-Connecting-IP, X-Real-IP, Forwarded, and True-Client-IP;
  • stream responses on /api/v1/module-realtime/* without buffering (Server-Sent Events);
  • set no request-body limit below 32 MiB; the API enforces its own ceilings;
  • forward the storage hostname to https://object-storage:8443, verifying it against /etc/ekklesia/tls/ca.crt with server name object-storage, and pass the original Host through unchanged, or every signed URL fails (ADR 0031).
  • reach object-storage over HTTP/1.1 only, and own the storage hostname's CORS as described in Storage delivery.

The proxy's address must be fixed and must never be handed to anything else. With Docker, give its network an explicit subnet, limit ip_range to part of it, and fix the proxy's address outside that range, as compose.proxy.yaml does.

Fixed addresses on edge

compose.proxy.yaml gives edge the subnet 10.249.250.0/24 and lets Docker assign dynamic addresses only from its lower half (10.249.250.0/25). The proxy (.254) and the tunnel connector (.253) take fixed addresses from the top of the upper half, which Docker never assigns to anything else. That is what makes "only the proxy" and "only the connector" expressible as /32 trust entries. If the subnet collides with one on your host, change EKKLESIA_EDGE_SUBNET, EKKLESIA_EDGE_DYNAMIC_RANGE, EKKLESIA_PROXY_ADDRESS, and EKKLESIA_TUNNEL_ADDRESS together, keeping the fixed addresses at the top and outside the dynamic range.

Cloudflare Tunnel

compose.tunnel.yaml runs cloudflared with a token, so the tunnel is managed in the Cloudflare dashboard. No Cloudflare API is called from here.

  1. In Cloudflare Zero Trust, under Networks → Tunnels, create a Cloudflared tunnel. Copy its token into /etc/ekklesia/tunnel.env as TUNNEL_TOKEN.
  2. Add a public hostname (shown as a published application route in newer dashboards) for the Ekklesia hostname. Service type HTTPS, URL proxy:443.
  3. Under that hostname's TLS settings, set Origin Server Name to the same Ekklesia hostname, and leave No TLS Verify off. The connector then verifies the proxy's certificate for that name.
  4. Add a public hostname for the storage hostname, the same way, with its own origin server name. In the split topology, add one for the Dashboard's hostname too: three in all, each with its own origin server name.
  5. Start with all three files. The proxy honors CF-Connecting-IP only from the connector's fixed address. From any other peer it is stripped.

The proxy's certificate must still be publicly trusted, and an HTTP-01 challenge cannot reach the proxy through a tunnel, because Cloudflare terminates TLS at its edge. A tunnel-only installation therefore uses DNS-01. An installation that also serves local displays should give the same hostnames a LAN path to the proxy (split-horizon DNS), so displays do not depend on the Internet link.

DNS-01

For an installation off the public Internet, LAN-only, or tunnel-only. DNS-01 needs a DNS provider module, which the stock Caddy image does not contain:

  1. Build a Caddy image with the module (for example xcaddy build --with github.com/caddy-dns/cloudflare) and pin it by digest.
  2. Override the proxy's image: in an overlay of your own. Give it CLOUDFLARE_API_TOKEN (a token scoped to DNS edits for the zone) through a root-owned env file.
  3. Enable the commented acme_dns line in your Caddyfile.

Launch checklist

The ten items in Security § launch checklist mapped to this deployment:

# Item Where it is met
1 The proxy forwards the Dashboard and /api on the intended origin Caddyfile.example routes /, /api/*, /display/*, /assets/* on one hostname to api; the API serves the Dashboard itself. In the split topology, Caddyfile.split.example gives each hostname exactly its own paths and the exact trusted-origin CORS of ADR 0045 covers the rest.
2 The origin port is unreachable from the Internet The base publishes no port; only proxy publishes 80 and 443.
3 The origin port is unreachable from untrusted networks api is reachable only on Docker networks; EKKLESIA_PUBLISH_ADDRESS limits the proxy to one interface. Firewall the host itself.
4 The proxy overwrites the trusted client-IP header header_up X-Forwarded-For {client_ip} in Caddyfile.example.
5 Rotating forwarding headers cannot rotate the rate-limit bucket The same, plus stripping CF-Connecting-IP, X-Real-IP, Forwarded, and True-Client-IP; the API overwrites its internal header. Proven by the smoke test.
6 The proxy peer is the only trusted proxy compose.proxy.yaml sets EKKLESIA_CLIENT_ADDRESS_TRUSTED_PROXIES to the proxy's fixed /32.
7 Proxy logs record no cookies, credentials, bodies, or forwarding chains Caddy access logs are off in Caddyfile.example. If you enable them, exclude request headers.
8 PostgreSQL only from the API, over verified TLS postgres is only on internal data; pg_hba.conf rejects non-TLS; verify-full with EKKLESIA_DATABASE_CA. Proven by the smoke test.
9 The deployed login and logout flow keeps its contract Unchanged by the proxy; the smoke test signs in and out through it.
10 A supported way to create the first production identity create-initial-owner.js through migrate (The initial owner). Exercised by the smoke test.

Testing this deployment

  • pnpm deploy:config validates the base, base + proxy, and base + proxy + tunnel with the example env files. CI runs it in pnpm ci:verify.

  • bash scripts/check-image.sh builds the image and proves it is generic: no /opt/ekklesia/modules, no module template, no ekklesia.module.json outside installed dependencies, an empty root-owned /var/lib/ekklesia/modules, and EKKLESIA_MODULES_DIR set. It then builds again from a checkout with a populated modules/ directory and requires an identical runtime file system. CI runs it.

  • pnpm deploy:smoke (scripts/smoke-deploy.sh) runs the whole deployment end to end, against the generic image, empty module storage, and a disposable PostgreSQL and SeaweedFS that exist only for that run. It follows the first boot above and checks the following through the proxy:

    • readiness and the Dashboard;
    • a signed object URL fetched through the storage hostname, and the same image loaded twice over HTTP/2, the second time as a conditional request that must be a clean 304;
    • storage CORS: exactly the allowed origins, GET and HEAD only, preflights answered by the proxy, and none of the object store's own CORS headers;
    • the client-address chain, both from an ordinary peer and from the tunnel connector's address;
    • initial-owner sign-in and sign-out;
    • that the API's database connection is TLS 1.3, and that it is refused by a foreign CA and by a wrong host name;
    • Doctor.

    EKKLESIA_SMOKE_TOPOLOGY=split runs the split topology instead, on the same generic image, and also proves Installed modules end to end:

    • the image boots with no module;
    • the repository's module template, packaged as an Ekklesia Module Package v1 by the canonical packager, is verified by the image itself and installed into module storage, its migration is planned and explicitly applied, the API restarts, and the module reads data it owns;
    • every hostname's served and refused paths, exact CORS for the Dashboard's origin and none for others;
    • a real headless Chromium (pinned by digest, driven over the DevTools protocol) signs in on the Dashboard's hostname, checks the session cookie lives only on the API's, mounts the module's UI from the API's origin and sees its own credentialed requests succeed, mounts its display on the API's hostname with its service worker, and signs out;
    • the module is removed and the API restarted: Core still runs, the module's routes are gone, migration planning and applying succeed and report its retained history, and its data is intact;
    • the exact same package is verified and installed again and the API restarted: its history validates strictly and it returns its previous data.

    CI runs both topologies on every change.

  • Test-only: the smoke test runs Caddy with its internal CA (local_certs), because a test run has no public hostname. Never use it in an installation; Caddyfile.example does not contain it.

The smoke test uses no module, so it does not exercise realtime streams or large uploads. Through this same proxy configuration, a scratch module confirmed both:

  • a Server-Sent Event crossed the proxy in milliseconds;
  • a 32 MiB streamed upload passed with and without Content-Length;
  • one byte more was refused by the API's own 413 REQUEST_BODY_TOO_LARGE.

The production image

Dockerfile builds one image for two jobs: the long-running api service, and every operator-run command (migrations, Doctor, initial-owner creation, module enablement). There is no second image and no second build.

What it builds

The build context is this repository, and the image is the same for every installation (ADR 0048). It contains zero feature modules: installed modules live in module storage outside it (Installed modules). No input is specific to an installation: no module list, no lockfile of its own, no branding, no configuration.

A modules/ directory in the checkout never reaches the image. .dockerignore keeps it out of the build context, and the build refuses a context that still contains one. The module template is a workspace package, so it is built and validated with everything else, but neither its files nor its dependencies reach the runtime image. scripts/check-image.sh proves all of this on every change (Testing this deployment).

Stage Does
base node:24.18.0-trixie-slim, pinned by its multi-architecture index digest. Debian, so glibc, because module dependencies such as sharp ship glibc prebuilt binaries. Every stage starts here.
toolchain Enables pnpm through Corepack, at exactly the root packageManager version.
manifests Collects the lockfile, workspace configuration, .npmrc, and the package.json of every platform package (not the module template's) at its repository path.
build Refuses a context containing modules/, then pnpm install --frozen-lockfile, pnpm boundaries, pnpm modules:validate, pnpm migrations:validate, and pnpm build. Never uses host node_modules: .dockerignore excludes them.
prod-deps The same lockfile with --prod, for the platform packages only: production dependencies only.
runtime Production dependencies plus built outputs, owned by root and run as the unprivileged node user (uid 1000), and an empty, root-owned /var/lib/ekklesia/modules. No pnpm, no compiler, no curl.

The runtime keeps the repository layout at /opt/ekklesia, because that layout is how the platform finds its files:

  • the API takes the repository root as three directories above apps/api/dist/, and reads package.json and packages/module-sdk/package.json beneath it;
  • every CLI takes its working directory as the root and also reads Core's migrations in core/drizzle/, so commands must run from /opt/ekklesia, the image's WORKDIR;
  • the Dashboard and display host files are read from surfaces/shell/dist/browser/;
  • installed modules are read from EKKLESIA_MODULES_DIR, /var/lib/ekklesia/modules, never from under /opt/ekklesia. With nothing mounted there, the image runs with no modules.

Building

docker build \
  --file deploy/Dockerfile \
  --build-arg VERSION=0.2.4 \
  --build-arg REVISION="$(git rev-parse HEAD)" \
  --build-arg SOURCE=https://git.example.org/your-org/ekklesia \
  --tag ekklesia:0.2.4 \
  .

The three build arguments become the OCI labels org.opencontainers.image.version, .revision, and .source. They have no defaults: the repository cannot know an installation's Git host. The image also records its base image name and digest. BuildKit is required (Docker's default builder); set DOCKER_BUILDKIT=1 if your Docker CLI has no buildx plugin.

No secret is a build argument, and none enters any layer. .env files are excluded from the build context entirely, and no registry credentials are needed.

The runtime an installed module targets

An installed module runs on the image's Node.js, inside the API process, from its own directory (ADR 0048):

  • Node.js 24.18.0 on Debian 13, so glibc, for the architecture the image was built for. A native dependency such as sharp must be built or downloaded for exactly that.
  • Its own dependencies only. Node resolves a module's imports from the module's directory, and the platform's node_modules is never on that path. A module whose server code imports something its directory does not contain stops the API from starting, with a sanitized MODULE_RUNTIME_IMPORT_FAILED.
  • The platform's exact Drizzle version, for now. Core runs a module's queries through its own Drizzle instance over tables the module defined with its own copy, so the two copies must be interchangeable. That is a constraint of the current implementation, not a permanent contract.

The repository's boundary checks (pnpm boundaries) prove only the source modules in a workspace. An installed module's conformance is its own CI's job, so check a module before installing it: validate-modules.js (Operator commands) validates what can be checked on an installed directory, and the API's startup refuses anything it cannot load.

Configuration

Configuration comes only from the container environment, never from a file inside the image. The image sets exactly four variables itself:

Variable Image value Why
NODE_ENV production Stated rather than relied on. Anything other than development already validates as production.
EKKLESIA_API_HOST 0.0.0.0 Reachable from other containers. Outside a container the platform default stays 127.0.0.1.
EKKLESIA_API_PORT 3000 The platform default, made explicit; EXPOSE 3000.
EKKLESIA_MODULES_DIR /var/lib/ekklesia/modules Where installed modules are read from (ADR 0048). Outside a container, unset means the checkout's own modules/. Compose states the same value beside the mount.

Everything else is the ordinary host configuration in API § Required configuration, supplied by the reference deployment's ekklesia.env. EKKLESIA_DATABASE_CA is the CA's PEM text itself, on one double-quoted line with \n escapes. Compose expands those escapes; plain docker run --env-file does not, which is why every command here goes through Compose.

Never publish the API's port to the host or the Internet. Only the proxy publishes ports.

Process, signals, and the read-only posture

compose.yaml runs api and migrate with init: true, read_only: true, tmpfs: /tmp, cap_drop: [ALL], and no-new-privileges. That is the supported production posture:

  • init: true runs a minimal init as PID 1 that forwards signals and reaps children, so docker stop reaches the API's own SIGTERM handler promptly. The API then drains in-flight requests, stops modules in reverse order, and closes its pool.
  • Read-only: the image was verified under exactly these flags. The API reached healthy, and the migration and Doctor commands ran. Nothing wrote to /tmp, which stays mounted as headroom for module code that needs scratch space. The installation, /home/node included, is not writable, and neither is module storage, which is mounted read-only. Anything that needs a writable path other than /tmp is a defect to report, not to work around.

Health

The image's HEALTHCHECK calls GET /api/v1/health/ready on 127.0.0.1 with Node's built-in fetch, every 30 seconds after a 30-second start period, with three retries. Readiness means the database answered within 2 seconds and the instance is not shutting down; see API § Health. Because it probes 127.0.0.1, keep EKKLESIA_API_HOST at 0.0.0.0 (or any address that includes loopback) inside the container. The API trusts only the proxy's address for client-address headers, so the loopback probe is an ordinary direct peer.

An external monitor can use the same endpoint through the proxy. Use /api/v1/health/live to ask only whether the process is serving.

Operator commands

The same image runs every CLI, directly with node core/dist/cli/<command>.js, through the migrate service. The pnpm scripts of the same names rebuild first and read .env, so they are for a development checkout, not for the image.

migrate has profiles: ["ops"], so up never starts it. It never restarts, has no health check, and its default command is the read-only plan. Nothing applies migrations automatically.

Run operator commands with the same Compose files as the stack. COMPOSE_FILE in deploy/.env (set in env/compose.env.example) does that for every plain docker compose command below. It matters for Doctor in particular: Doctor checks the client-address configuration of the process it runs in. The proxy overlay gives migrate the API's trusted-proxy settings, so Doctor reports what the API runs. Without the overlay, Doctor reports direct mode for an API that trusts its proxy. The running API's own value is shown by docker compose exec api printenv EKKLESIA_CLIENT_ADDRESS_MODE.

Task Command (from deploy/)
Plan migrations (read-only) docker compose run --rm migrate
Apply migrations docker compose run --rm migrate node core/dist/cli/apply-migrations.js
Doctor (read-only report) docker compose run --rm migrate node core/dist/cli/doctor.js
Create the initial owner (interactive) docker compose run --rm -it migrate node core/dist/cli/create-initial-owner.js --email [email protected]
Validate the installed modules (read-only) docker compose run --rm migrate node core/dist/cli/validate-modules.js
Verify a module package (read-only unless extracting) node core/dist/cli/verify-module-package.js <archive> --sha256-file <archive>.sha256 [--extract-to <new directory>], in a docker run of the image (Installed modules)
Enable or disable a module docker compose run --rm migrate node core/dist/cli/set-module-enabled.js --enable --module <id> (or --disable)

Every command reads installed modules from the same EKKLESIA_MODULES_DIR the API does. The plan, apply, Doctor, and initial-owner commands are exercised by the smoke test, and so are installing, removing, and restoring a module; module validation and enablement are not.

Size

The image is about 378 MB, and it contains no module: the Debian slim Node base (about 230 MB), production dependencies (about 141 MB), and built outputs (about 6 MB). A rebuild that reuses a warm pnpm store cache can come out about 21 MB larger. When esbuild's install script runs, it hard-links its bin/esbuild to the native binary it ships beside it; when pnpm restores esbuild from the store's cached post-install state instead, that file is a second full copy. Both are esbuild, one of the unused tools below.

About 57 MB of the production dependencies are development tools that no runtime code imports: vitest, vite, drizzle-kit, tsx, esbuild, and their native bindings. They arrive as optional peer dependencies of Better Auth (and esbuild through the module UI builder), which pnpm resolves because the workspace already has them. Excluding them means changing dependency resolution and the lockfile, which is separate work.