feat: Ekklesia Module Package v1, the canonical packager and verifier (ADR 0049)
CI / verify (pull_request) Successful in 57s
CI / infrastructure-smoke (pull_request) Successful in 35s
CI / migration-smoke (pull_request) Successful in 1m42s
CI / image-build (pull_request) Successful in 29s
CI / deploy-smoke (same-origin) (pull_request) Successful in 51s
CI / deploy-smoke (split) (pull_request) Failing after 54s

A separately versioned module is now distributed as one deterministic
.tar.gz and its SHA-256, produced by an Ekklesia-owned packager and checked by
Core before anything is installed. This slice implements the portable `any`
target, with the module template as the end-to-end proof. ADR 0049 is
accepted.

@ekklesia/contracts: the Module Package v1 contract. The strict
ekklesia.package.json schema (packageFormat 1, identity, target, runtime,
provenance, incorporated Ekklesia packages, strict transitional facts, an
exact byte-wise-sorted file list with size and SHA-256), target names, and the
generated runtime package.json.

@ekklesia/module-pack (new, tooling): `ekklesia-pack-module --module <dir>
--target any --out <dir> --source-revision <commit>`.
- Bundles server, schema, and jobs with esbuild, SDK, contracts, and Drizzle
  included; only Node built-ins remain imports. Any warning, unresolved
  import, external dependency, or native or platform-specific package fails.
- Copies browser artifacts as built, after proving them current, and the
  migrations; generates package.json and ekklesia.package.json; never ships
  src, tests, declarations, source maps, or workspace ranges.
- THIRD_PARTY_NOTICES: every third-party package traced from the bundles'
  own inputs, credited once with its declared licence and every licence,
  copying, copyright, and notice file it ships. A missing licence fails; a
  missing text fails unless its standard text is complete without copyright
  lines (Apache-2.0, vendored, needed by drizzle-orm).
- Deterministic ustar and gzip: sorted paths, uid/gid 0, empty names, mtime
  0, 0755/0644, no links. Byte-identical across runs and machines.
- Writes only the archive and its .sha256; never deploys, links, or installs,
  so the workspace's install state is untouched.

Core: verifyModulePackage and `verify-module-package.js <archive>
[--sha256 | --sha256-file] [--extract-to <new dir>]`. Reads the archive in
memory and refuses links, special entries, setuid/setgid/sticky modes, unsafe
or escaping paths, and a second top-level directory; checks metadata, every
file, identity, target, Node.js range, SDK, and the transitional Drizzle
agreement; runs the platform's module and migration discovery; loads the
backend once in a forked process with an empty environment; only then
extracts, with normalized modes, into a new directory. Errors are sanitized.

@ekklesia/module-ui-build: analyzeModuleUi reports each artifact's inputs
and whether the built file is current. The nested stylesheet build now works
from the module directory, so artifacts no longer depend on the caller's
working directory.

The split deploy smoke builds the package with the canonical packager
(scripts/lib/build-module-package.Dockerfile replaces the test-only pnpm
deploy packaging), verifies it with the same generic image, installs the
verified tree, removes it, and reinstalls the exact same archive.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-10-07 12:44:54 -05:00
co-authored by Claude Opus 5.5
parent 45d6506c10
commit 35ef31df9f
45 changed files with 4876 additions and 151 deletions
+33 -12
View File
@@ -142,12 +142,16 @@ Installed modules live on the host, in module storage: `/var/lib/ekklesia/module
and `migrate`, which is where `EKKLESIA_MODULES_DIR` points. Ekklesia never writes there.
Installing, updating, or removing a module never needs a new image, only a restart.
**What belongs there.** One directory per module, each a self-contained installed module: its
`package.json`, `ekklesia.module.json`, built artifacts, migrations, and the production
dependencies its server code imports, in its own `node_modules`
**What belongs there.** One directory per module, each extracted from that module's **Ekklesia
Module Package**
([ADR 0049](../docs/architecture/decisions/0049-module-package-v1.md)): one archive,
`<id>-<version>-<target>.tar.gz`, and the SHA-256 published beside it. A package holds the module's
generated `package.json`, its manifest, `ekklesia.package.json`, bundled backend code, built browser
artifacts, migrations, and `THIRD_PARTY_NOTICES`
([Module development](../docs/module-development/README.md#an-installed-module-is-self-contained)).
Copying a module's `dist/` alone is not enough. Files are owned by root and readable by everyone
(directories 755, files 644), so the API's unprivileged user can read them and never change them.
The digest proves the archive is the one you were given; it does not prove who published it.
**Nothing else belongs there.** Every non-hidden directory must be a valid module, or the API does
not start. That includes `lost+found`, so if module storage is a file system of its own, mount it
@@ -159,12 +163,28 @@ never loads or unloads a module while it runs. Change module storage only while
it reads browser artifacts from disk as it serves them, so files changed under a running API would
serve one version's browser code beside another's server code.
To **install or update** a module, with `<id>` its module id:
To **install or update** a module, with `<id>` its module id and the package and its `.sha256` in
the current directory, first **verify** it with the image's own verifier. It checks the archive's
digest, its safety (no links, special files, or escaping paths), every file's digest, the package's
identity, target, and Node.js range against this platform, runs the platform's module and migration
validation, and loads the backend once in a short-lived process with no environment. Only then does
it write the verified module directory, to a new directory outside module storage:
```sh
mkdir -m 777 staging # writable by the image's unprivileged user; nothing else is
docker run --rm --network none --read-only --tmpfs /tmp \
-v "$PWD:/package:ro" -v "$PWD/staging:/staging" \
--entrypoint node "$EKKLESIA_IMAGE" core/dist/cli/verify-module-package.js \
/package/<id>-<version>-any.tar.gz --sha256-file /package/<id>-<version>-any.tar.gz.sha256 \
--extract-to /staging/<id>
```
Then install the verified directory:
```sh
docker compose stop api
# Stage under a hidden name, which discovery ignores, then rename it into place in one step.
sudo cp -R ./<id> /var/lib/ekklesia/modules/.incoming-<id>
sudo cp -R staging/<id> /var/lib/ekklesia/modules/.incoming-<id>
sudo chown -R root:root /var/lib/ekklesia/modules/.incoming-<id>
sudo chmod -R a+rX,go-w /var/lib/ekklesia/modules/.incoming-<id>
sudo mv /var/lib/ekklesia/modules/<id> /var/lib/ekklesia/modules/.previous-<id> # update only
@@ -188,7 +208,8 @@ is information, not a problem, and it blocks no other migration. To stop a modul
it, disable it instead (`set-module-enabled.js --disable`, [Operator commands](#operator-commands)).
There is no purge: deleting a module's data is a separate operation that does not exist yet.
To **restore** a removed module, install it again as above, at the same version or a newer one. It
To **restore** a removed module, verify and install its package again as above, at the same version
or a newer one. It
finds its data, its settings, and its history where it left them. Its history is strict again: a
package that no longer contains a migration that was applied, or contains one with different
contents, is refused by the migration plan, so an older version cannot be installed over newer data.
@@ -409,10 +430,9 @@ deployment:
instead, on the **same generic image**, and also proves [Installed modules](#installed-modules)
end to end:
- the image boots with no module;
- the repository's module template, packaged as a self-contained test module
(`scripts/lib/test-module-package.Dockerfile`, test-only and not a package format), is installed
into module storage, its migration is planned and explicitly applied, the API restarts, and the
module reads data it owns;
- the repository's module template, packaged as an Ekklesia Module Package v1 by the canonical
packager, is verified by the image itself and installed into module storage, its migration is
planned and explicitly applied, the API restarts, and the module reads data it owns;
- every hostname's served and refused paths, exact CORS for the Dashboard's origin and none for
others;
- a real headless Chromium (pinned by digest, driven over the DevTools protocol) signs in on the
@@ -421,8 +441,8 @@ deployment:
API's hostname with its service worker, and signs out;
- the module is removed and the API restarted: Core still runs, the module's routes are gone,
migration planning and applying succeed and report its retained history, and its data is intact;
- the module is restored and the API restarted: its history validates strictly and it returns its
previous data.
- the exact same package is verified and installed again and the API restarted: its history
validates strictly and it returns its previous data.
CI runs both topologies on every change.
- **Test-only:** the smoke test runs Caddy with its internal CA (`local_certs`), because a test run
@@ -586,6 +606,7 @@ reports direct mode for an API that trusts its proxy. The running API's own valu
| Doctor (read-only report) | `docker compose run --rm migrate node core/dist/cli/doctor.js` |
| Create the initial owner (interactive) | `docker compose run --rm -it migrate node core/dist/cli/create-initial-owner.js --email owner@example.org` |
| Validate the installed modules (read-only) | `docker compose run --rm migrate node core/dist/cli/validate-modules.js` |
| Verify a module package (read-only unless extracting) | `node core/dist/cli/verify-module-package.js <archive> --sha256-file <archive>.sha256 [--extract-to <new directory>]`, in a `docker run` of the image ([Installed modules](#installed-modules)) |
| Enable or disable a module | `docker compose run --rm migrate node core/dist/cli/set-module-enabled.js --enable --module <id>` (or `--disable`) |
Every command reads installed modules from the same `EKKLESIA_MODULES_DIR` the API does. The plan,