feat: Ekklesia Module Package v1, the canonical packager and verifier (ADR 0049)
CI / verify (pull_request) Successful in 57s
CI / infrastructure-smoke (pull_request) Successful in 35s
CI / migration-smoke (pull_request) Successful in 1m42s
CI / image-build (pull_request) Successful in 29s
CI / deploy-smoke (same-origin) (pull_request) Successful in 51s
CI / deploy-smoke (split) (pull_request) Failing after 54s

A separately versioned module is now distributed as one deterministic
.tar.gz and its SHA-256, produced by an Ekklesia-owned packager and checked by
Core before anything is installed. This slice implements the portable `any`
target, with the module template as the end-to-end proof. ADR 0049 is
accepted.

@ekklesia/contracts: the Module Package v1 contract. The strict
ekklesia.package.json schema (packageFormat 1, identity, target, runtime,
provenance, incorporated Ekklesia packages, strict transitional facts, an
exact byte-wise-sorted file list with size and SHA-256), target names, and the
generated runtime package.json.

@ekklesia/module-pack (new, tooling): `ekklesia-pack-module --module <dir>
--target any --out <dir> --source-revision <commit>`.
- Bundles server, schema, and jobs with esbuild, SDK, contracts, and Drizzle
  included; only Node built-ins remain imports. Any warning, unresolved
  import, external dependency, or native or platform-specific package fails.
- Copies browser artifacts as built, after proving them current, and the
  migrations; generates package.json and ekklesia.package.json; never ships
  src, tests, declarations, source maps, or workspace ranges.
- THIRD_PARTY_NOTICES: every third-party package traced from the bundles'
  own inputs, credited once with its declared licence and every licence,
  copying, copyright, and notice file it ships. A missing licence fails; a
  missing text fails unless its standard text is complete without copyright
  lines (Apache-2.0, vendored, needed by drizzle-orm).
- Deterministic ustar and gzip: sorted paths, uid/gid 0, empty names, mtime
  0, 0755/0644, no links. Byte-identical across runs and machines.
- Writes only the archive and its .sha256; never deploys, links, or installs,
  so the workspace's install state is untouched.

Core: verifyModulePackage and `verify-module-package.js <archive>
[--sha256 | --sha256-file] [--extract-to <new dir>]`. Reads the archive in
memory and refuses links, special entries, setuid/setgid/sticky modes, unsafe
or escaping paths, and a second top-level directory; checks metadata, every
file, identity, target, Node.js range, SDK, and the transitional Drizzle
agreement; runs the platform's module and migration discovery; loads the
backend once in a forked process with an empty environment; only then
extracts, with normalized modes, into a new directory. Errors are sanitized.

@ekklesia/module-ui-build: analyzeModuleUi reports each artifact's inputs
and whether the built file is current. The nested stylesheet build now works
from the module directory, so artifacts no longer depend on the caller's
working directory.

The split deploy smoke builds the package with the canonical packager
(scripts/lib/build-module-package.Dockerfile replaces the test-only pnpm
deploy packaging), verifies it with the same generic image, installs the
verified tree, removes it, and reinstalls the exact same archive.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-10-07 12:44:54 -05:00
co-authored by Claude Opus 5.5
parent 45d6506c10
commit 35ef31df9f
45 changed files with 4876 additions and 151 deletions
@@ -0,0 +1,31 @@
# Builds the repository's module template and packages it with the canonical Ekklesia packager
# (`@ekklesia/module-pack`, ADR 0049), in a clean container from a clean copy of the checkout. The
# deploy smoke test uses it because a CI job that drives Docker has no Node.js of its own. The
# packaging itself is exactly what a module's CI runs; this file only provides the environment.
#
# DOCKER_BUILDKIT=1 docker build --file scripts/lib/build-module-package.Dockerfile \
# --build-arg SOURCE_REVISION=<full commit id> --output type=local,dest=<directory> .
#
# writes <directory>/example-<version>-any.tar.gz and its .sha256 line. No image is produced.
#
# The same pinned Node.js base as deploy/Dockerfile, so the package is built by the runtime it is
# installed into.
FROM node:24.18.0-trixie-slim@sha256:ae91dcc111a68c9d2d81ff2a17bda61be126426176fde6fe7d08ab13b7f50573 AS build
ENV COREPACK_ENABLE_DOWNLOAD_PROMPT=0 \
PNPM_HOME=/pnpm \
TURBO_TELEMETRY_DISABLED=1 \
DO_NOT_TRACK=1
WORKDIR /opt/source
COPY package.json ./
RUN corepack enable && corepack install
COPY . .
RUN --mount=type=cache,id=ekklesia-pnpm-store,target=/pnpm/store \
pnpm install --frozen-lockfile --store-dir /pnpm/store
RUN pnpm exec turbo run build --filter=@ekklesia/module-template --filter=@ekklesia/module-pack
ARG SOURCE_REVISION
RUN test -n "$SOURCE_REVISION" && \
node packages/module-pack/bin/ekklesia-pack-module.mjs --module templates/module --target any \
--out /package --source-revision "$SOURCE_REVISION"
FROM scratch AS package
COPY --from=build /package/ /
@@ -1,50 +0,0 @@
# TEST-ONLY. Builds the repository's module template as one self-contained installed-module directory
# for scripts/smoke-deploy.sh, which installs it into a deployment's module storage (ADR 0048). It is
# not a module package format and defines none: the archive format, signing, and transport of real
# module packages are deferred. No image is produced; the build's only output is the directory:
#
# DOCKER_BUILDKIT=1 docker build --file scripts/lib/test-module-package.Dockerfile \
# --output type=local,dest=<directory> .
#
# writes <directory>/example: the module's package.json, manifest, built artifacts, migrations, and a
# node_modules holding exactly its production dependencies, including its own copies of the Module
# SDK, the contracts, and Drizzle. Nothing in it resolves to anything outside it, because the
# platform lends an installed module nothing.
#
# The same pinned Node.js base as deploy/Dockerfile, so the package targets the runtime it is
# installed into. Drizzle comes from the platform's own lockfile, which is what keeps it the exact
# version the platform uses: a constraint of the current implementation (ADR 0048), not a contract.
FROM node:24.18.0-trixie-slim@sha256:ae91dcc111a68c9d2d81ff2a17bda61be126426176fde6fe7d08ab13b7f50573 AS build
ENV COREPACK_ENABLE_DOWNLOAD_PROMPT=0 \
PNPM_HOME=/pnpm \
TURBO_TELEMETRY_DISABLED=1 \
DO_NOT_TRACK=1
WORKDIR /opt/source
COPY package.json ./
RUN corepack enable && corepack install
COPY . .
RUN --mount=type=cache,id=ekklesia-pnpm-store,target=/pnpm/store \
pnpm install --frozen-lockfile --store-dir /pnpm/store
RUN pnpm exec turbo run build --filter=@ekklesia/module-template
# `--legacy` deploys from this workspace without switching it to injected workspace packages; the
# Module SDK and every other workspace dependency are still copied into the package, built.
RUN --mount=type=cache,id=ekklesia-pnpm-store,target=/pnpm/store \
set -eu; \
pnpm --filter @ekklesia/module-template deploy --prod --legacy --store-dir /pnpm/store \
/package/example; \
rm -f /package/example/README.md; \
# pnpm's hoisted self-reference points back into this workspace; an installed module has none.
rm -f /package/example/node_modules/.pnpm/node_modules/@ekklesia/module-template; \
escaping=""; \
for link in $(find /package/example -type l); do \
case "$(realpath -m "$link")" in \
/package/example/*) ;; \
*) escaping="$escaping $link" ;; \
esac; \
done; \
if [ -n "$escaping" ]; then echo "Links escape the package:$escaping" >&2; exit 1; fi; \
chmod -R a+rX,go-w /package/example; \
cd /package/example && node -e 'import("./dist/server/index.js").then((m) => { if (m.default.id !== "example") process.exit(1); })'
FROM scratch AS package
COPY --from=build /package/ /
+48 -22
View File
@@ -15,13 +15,15 @@
# ADR 0048 end to end with that same image:
#
# - it boots with no modules;
# - the repository's module template, packaged as one self-contained directory
# (scripts/lib/test-module-package.Dockerfile, TEST-ONLY), is installed into module storage, its
# migrations are planned and explicitly applied, and the API is restarted;
# - the repository's module template is packaged by the canonical packager as an Ekklesia Module
# Package v1 (ADR 0049; scripts/lib/build-module-package.Dockerfile only provides the build
# environment), verified by the same image's own verifier, extracted to staging, installed into
# module storage, its migrations planned and explicitly applied, and the API restarted;
# - a real headless Chromium drives the Dashboard, the module, its display, and sign-out;
# - the module is removed and the API restarted: Core still runs, migration planning reports the
# module's retained history, and its data survives;
# - the module is restored and the API restarted: it returns with its previous data.
# - the exact same package is verified and installed again, and the API restarted: the module
# returns with its previous data.
#
# EKKLESIA_SMOKE_IMAGE uses the given image instead of building one. It must be a generic image
# (ADR 0048): modules come from module storage, never from the image.
@@ -179,7 +181,7 @@ cleanup() {
# job itself runs as root), so only the staged copy on the Docker host needs this.
if [[ "$daemon_work" == "$work" ]]; then
docker run --rm --entrypoint rm -v "$work:/work" "$postgres_image" -rf /work/tls /work/env \
/work/modules >/dev/null 2>&1 || true
/work/modules /work/staging >/dev/null 2>&1 || true
else
docker run --rm --entrypoint rm -v /tmp:/host-tmp "$postgres_image" -rf "/host-tmp/${project}" \
>/dev/null 2>&1 || true
@@ -213,15 +215,20 @@ if [[ -z "$image" ]]; then
fi
if [[ "$topology" == split ]]; then
# TEST-ONLY: the module template as one self-contained installed-module directory, built outside
# the platform image and installed into module storage below. Never a second platform image.
step "Packaging the module template as a self-contained test module"
DOCKER_BUILDKIT=1 docker build --quiet --file scripts/lib/test-module-package.Dockerfile \
--label "${EKKLESIA_CI_LABEL}=${project}" \
# The module template as a Module Package v1, from the canonical packager. Never a second platform
# image: the package is installed into module storage below, after the image verifies it.
step "Packaging the module template with the canonical Ekklesia packager"
source_revision="${GITHUB_SHA:-$(git rev-parse HEAD 2>/dev/null || true)}"
[[ "$source_revision" =~ ^[0-9a-f]{40}$ ]] || fail "no source revision to record in the package."
DOCKER_BUILDKIT=1 docker build --quiet --file scripts/lib/build-module-package.Dockerfile \
--label "${EKKLESIA_CI_LABEL}=${project}" --build-arg SOURCE_REVISION="$source_revision" \
--output "type=local,dest=${work}/module-package" . >/dev/null ||
fail "the test module package did not build."
[[ -f "$work/module-package/example/ekklesia.module.json" ]] ||
fail "the test module package has no manifest."
fail "the module package did not build."
package_archive="$(cd "$work/module-package" && ls -- *-any.tar.gz)"
[[ "$package_archive" =~ ^example-[0-9]+\.[0-9]+\.[0-9]+-any\.tar\.gz$ &&
-f "$work/module-package/${package_archive}.sha256" ]] ||
fail "the packager did not produce exactly one archive and its digest."
echo "ok packaged ${package_archive} ($(wc -c <"$work/module-package/$package_archive") bytes, sha256 $(cut -d' ' -f1 "$work/module-package/${package_archive}.sha256"))"
fi
# The compose project directory: its relative bind mount (postgres/pg_hba.conf) resolves here.
@@ -232,6 +239,13 @@ stage deploy
# `up`: Compose refuses to start `api` or `migrate` without it, and never creates it.
mkdir -m 755 "$work/modules"
stage modules
if [[ "$topology" == split ]]; then
stage module-package
# TEST-ONLY staging the verifier extracts into, as the image's unprivileged user. Nothing here is
# module storage: the install step below moves a verified tree into module storage as root.
mkdir -m 777 "$work/staging"
stage staging
fi
step "Creating the installation-private CA and server certificates"
init_status=0
@@ -420,15 +434,27 @@ module_storage() {
"$postgres_image" -ec "$1"
}
# install_test_module: stages the package under a hidden name, which discovery ignores, then renames
# it into place in one step, owned by root and readable by the API's unprivileged user.
# install_test_module: verifies the package with the generic image's own verifier, which writes the
# verified module directory to staging, then moves it into module storage under a hidden name, which
# discovery ignores, and renames it into place in one step, owned by root and readable by the API's
# unprivileged user. Nothing unverified ever reaches module storage.
install_test_module() {
tar -C "$work/module-package" -cf - example |
module_storage 'rm -rf /modules/.incoming && mkdir /modules/.incoming &&
tar -xf - -C /modules/.incoming && chown -R 0:0 /modules/.incoming/example &&
chmod -R a+rX,go-w /modules/.incoming/example &&
mv /modules/.incoming/example /modules/example && rmdir /modules/.incoming' ||
fail "the test module could not be installed into module storage."
local verified
verified="$(
docker run --rm --network none --read-only --tmpfs /tmp --cap-drop ALL \
--security-opt no-new-privileges \
-v "$daemon_work/module-package:/package:ro" -v "$daemon_work/staging:/staging" \
--entrypoint node "$image" core/dist/cli/verify-module-package.js "/package/${package_archive}" \
--sha256-file "/package/${package_archive}.sha256" --extract-to /staging/example 2>&1
)" || fail "the module package did not verify: ${verified}"
echo "ok ${verified%%$'\n'*}"
docker run --rm --network none --entrypoint sh -v "$daemon_work/modules:/modules" \
-v "$daemon_work/staging:/staging" "$postgres_image" -ec '
rm -rf /modules/.incoming-example && cp -R /staging/example /modules/.incoming-example &&
rm -rf /staging/example && chown -R 0:0 /modules/.incoming-example &&
chmod -R a+rX,go-w /modules/.incoming-example &&
mv /modules/.incoming-example /modules/example' ||
fail "the verified module could not be installed into module storage."
}
# restart_api <command...>: stops the API, runs the command, and starts the API again. A controlled
@@ -590,7 +616,7 @@ if [[ "$topology" == split ]]; then
fail "the removed module's data did not survive."
echo "ok the module is gone, Core runs, its history is retained, and its data survives"
step "Restoring the module and restarting: it returns with its previous data"
step "Reinstalling the exact same package and restarting: the module returns with its previous data"
restart_api install_test_module
plan="$(plan_output)"
grep -q '^- \[applied\] example:0000_initial ' <<<"$plan" ||