feat: Ekklesia Module Package v1, the canonical packager and verifier (ADR 0049)
CI / verify (pull_request) Successful in 57s
CI / infrastructure-smoke (pull_request) Successful in 35s
CI / migration-smoke (pull_request) Successful in 1m42s
CI / image-build (pull_request) Successful in 29s
CI / deploy-smoke (same-origin) (pull_request) Successful in 51s
CI / deploy-smoke (split) (pull_request) Failing after 54s

A separately versioned module is now distributed as one deterministic
.tar.gz and its SHA-256, produced by an Ekklesia-owned packager and checked by
Core before anything is installed. This slice implements the portable `any`
target, with the module template as the end-to-end proof. ADR 0049 is
accepted.

@ekklesia/contracts: the Module Package v1 contract. The strict
ekklesia.package.json schema (packageFormat 1, identity, target, runtime,
provenance, incorporated Ekklesia packages, strict transitional facts, an
exact byte-wise-sorted file list with size and SHA-256), target names, and the
generated runtime package.json.

@ekklesia/module-pack (new, tooling): `ekklesia-pack-module --module <dir>
--target any --out <dir> --source-revision <commit>`.
- Bundles server, schema, and jobs with esbuild, SDK, contracts, and Drizzle
  included; only Node built-ins remain imports. Any warning, unresolved
  import, external dependency, or native or platform-specific package fails.
- Copies browser artifacts as built, after proving them current, and the
  migrations; generates package.json and ekklesia.package.json; never ships
  src, tests, declarations, source maps, or workspace ranges.
- THIRD_PARTY_NOTICES: every third-party package traced from the bundles'
  own inputs, credited once with its declared licence and every licence,
  copying, copyright, and notice file it ships. A missing licence fails; a
  missing text fails unless its standard text is complete without copyright
  lines (Apache-2.0, vendored, needed by drizzle-orm).
- Deterministic ustar and gzip: sorted paths, uid/gid 0, empty names, mtime
  0, 0755/0644, no links. Byte-identical across runs and machines.
- Writes only the archive and its .sha256; never deploys, links, or installs,
  so the workspace's install state is untouched.

Core: verifyModulePackage and `verify-module-package.js <archive>
[--sha256 | --sha256-file] [--extract-to <new dir>]`. Reads the archive in
memory and refuses links, special entries, setuid/setgid/sticky modes, unsafe
or escaping paths, and a second top-level directory; checks metadata, every
file, identity, target, Node.js range, SDK, and the transitional Drizzle
agreement; runs the platform's module and migration discovery; loads the
backend once in a forked process with an empty environment; only then
extracts, with normalized modes, into a new directory. Errors are sanitized.

@ekklesia/module-ui-build: analyzeModuleUi reports each artifact's inputs
and whether the built file is current. The nested stylesheet build now works
from the module directory, so artifacts no longer depend on the caller's
working directory.

The split deploy smoke builds the package with the canonical packager
(scripts/lib/build-module-package.Dockerfile replaces the test-only pnpm
deploy packaging), verifies it with the same generic image, installs the
verified tree, removes it, and reinstalls the exact same archive.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-10-07 12:44:54 -05:00
co-authored by Claude Opus 5.5
parent 45d6506c10
commit 35ef31df9f
45 changed files with 4876 additions and 151 deletions
+48 -22
View File
@@ -15,13 +15,15 @@
# ADR 0048 end to end with that same image:
#
# - it boots with no modules;
# - the repository's module template, packaged as one self-contained directory
# (scripts/lib/test-module-package.Dockerfile, TEST-ONLY), is installed into module storage, its
# migrations are planned and explicitly applied, and the API is restarted;
# - the repository's module template is packaged by the canonical packager as an Ekklesia Module
# Package v1 (ADR 0049; scripts/lib/build-module-package.Dockerfile only provides the build
# environment), verified by the same image's own verifier, extracted to staging, installed into
# module storage, its migrations planned and explicitly applied, and the API restarted;
# - a real headless Chromium drives the Dashboard, the module, its display, and sign-out;
# - the module is removed and the API restarted: Core still runs, migration planning reports the
# module's retained history, and its data survives;
# - the module is restored and the API restarted: it returns with its previous data.
# - the exact same package is verified and installed again, and the API restarted: the module
# returns with its previous data.
#
# EKKLESIA_SMOKE_IMAGE uses the given image instead of building one. It must be a generic image
# (ADR 0048): modules come from module storage, never from the image.
@@ -179,7 +181,7 @@ cleanup() {
# job itself runs as root), so only the staged copy on the Docker host needs this.
if [[ "$daemon_work" == "$work" ]]; then
docker run --rm --entrypoint rm -v "$work:/work" "$postgres_image" -rf /work/tls /work/env \
/work/modules >/dev/null 2>&1 || true
/work/modules /work/staging >/dev/null 2>&1 || true
else
docker run --rm --entrypoint rm -v /tmp:/host-tmp "$postgres_image" -rf "/host-tmp/${project}" \
>/dev/null 2>&1 || true
@@ -213,15 +215,20 @@ if [[ -z "$image" ]]; then
fi
if [[ "$topology" == split ]]; then
# TEST-ONLY: the module template as one self-contained installed-module directory, built outside
# the platform image and installed into module storage below. Never a second platform image.
step "Packaging the module template as a self-contained test module"
DOCKER_BUILDKIT=1 docker build --quiet --file scripts/lib/test-module-package.Dockerfile \
--label "${EKKLESIA_CI_LABEL}=${project}" \
# The module template as a Module Package v1, from the canonical packager. Never a second platform
# image: the package is installed into module storage below, after the image verifies it.
step "Packaging the module template with the canonical Ekklesia packager"
source_revision="${GITHUB_SHA:-$(git rev-parse HEAD 2>/dev/null || true)}"
[[ "$source_revision" =~ ^[0-9a-f]{40}$ ]] || fail "no source revision to record in the package."
DOCKER_BUILDKIT=1 docker build --quiet --file scripts/lib/build-module-package.Dockerfile \
--label "${EKKLESIA_CI_LABEL}=${project}" --build-arg SOURCE_REVISION="$source_revision" \
--output "type=local,dest=${work}/module-package" . >/dev/null ||
fail "the test module package did not build."
[[ -f "$work/module-package/example/ekklesia.module.json" ]] ||
fail "the test module package has no manifest."
fail "the module package did not build."
package_archive="$(cd "$work/module-package" && ls -- *-any.tar.gz)"
[[ "$package_archive" =~ ^example-[0-9]+\.[0-9]+\.[0-9]+-any\.tar\.gz$ &&
-f "$work/module-package/${package_archive}.sha256" ]] ||
fail "the packager did not produce exactly one archive and its digest."
echo "ok packaged ${package_archive} ($(wc -c <"$work/module-package/$package_archive") bytes, sha256 $(cut -d' ' -f1 "$work/module-package/${package_archive}.sha256"))"
fi
# The compose project directory: its relative bind mount (postgres/pg_hba.conf) resolves here.
@@ -232,6 +239,13 @@ stage deploy
# `up`: Compose refuses to start `api` or `migrate` without it, and never creates it.
mkdir -m 755 "$work/modules"
stage modules
if [[ "$topology" == split ]]; then
stage module-package
# TEST-ONLY staging the verifier extracts into, as the image's unprivileged user. Nothing here is
# module storage: the install step below moves a verified tree into module storage as root.
mkdir -m 777 "$work/staging"
stage staging
fi
step "Creating the installation-private CA and server certificates"
init_status=0
@@ -420,15 +434,27 @@ module_storage() {
"$postgres_image" -ec "$1"
}
# install_test_module: stages the package under a hidden name, which discovery ignores, then renames
# it into place in one step, owned by root and readable by the API's unprivileged user.
# install_test_module: verifies the package with the generic image's own verifier, which writes the
# verified module directory to staging, then moves it into module storage under a hidden name, which
# discovery ignores, and renames it into place in one step, owned by root and readable by the API's
# unprivileged user. Nothing unverified ever reaches module storage.
install_test_module() {
tar -C "$work/module-package" -cf - example |
module_storage 'rm -rf /modules/.incoming && mkdir /modules/.incoming &&
tar -xf - -C /modules/.incoming && chown -R 0:0 /modules/.incoming/example &&
chmod -R a+rX,go-w /modules/.incoming/example &&
mv /modules/.incoming/example /modules/example && rmdir /modules/.incoming' ||
fail "the test module could not be installed into module storage."
local verified
verified="$(
docker run --rm --network none --read-only --tmpfs /tmp --cap-drop ALL \
--security-opt no-new-privileges \
-v "$daemon_work/module-package:/package:ro" -v "$daemon_work/staging:/staging" \
--entrypoint node "$image" core/dist/cli/verify-module-package.js "/package/${package_archive}" \
--sha256-file "/package/${package_archive}.sha256" --extract-to /staging/example 2>&1
)" || fail "the module package did not verify: ${verified}"
echo "ok ${verified%%$'\n'*}"
docker run --rm --network none --entrypoint sh -v "$daemon_work/modules:/modules" \
-v "$daemon_work/staging:/staging" "$postgres_image" -ec '
rm -rf /modules/.incoming-example && cp -R /staging/example /modules/.incoming-example &&
rm -rf /staging/example && chown -R 0:0 /modules/.incoming-example &&
chmod -R a+rX,go-w /modules/.incoming-example &&
mv /modules/.incoming-example /modules/example' ||
fail "the verified module could not be installed into module storage."
}
# restart_api <command...>: stops the API, runs the command, and starts the API again. A controlled
@@ -590,7 +616,7 @@ if [[ "$topology" == split ]]; then
fail "the removed module's data did not survive."
echo "ok the module is gone, Core runs, its history is retained, and its data survives"
step "Restoring the module and restarting: it returns with its previous data"
step "Reinstalling the exact same package and restarting: the module returns with its previous data"
restart_api install_test_module
plan="$(plan_output)"
grep -q '^- \[applied\] example:0000_initial ' <<<"$plan" ||