From 1119cc5fdf6260017340a8badf5853f07f98c27d Mon Sep 17 00:00:00 2001 From: ekklesia Date: Wed, 7 Oct 2026 10:13:34 -0500 Subject: [PATCH] docs: propose ADR 0049, Ekklesia Module Package v1 A separately versioned module is distributed as one immutable, self-contained, deterministic .tar.gz with a separately distributed SHA-256. The package contract is independent of hosting, registry, transport, and marketplace. ADR 0049 extends ADR 0048 with: - the package layout: one top-level module directory with a generated runtime package.json, the unchanged manifest, ekklesia.package.json, bundled backend entrypoints, built browser artifacts, migrations, THIRD_PARTY_NOTICES, and node_modules only for native dependencies; - backend bundling of all pure-JavaScript dependencies, with externals decided and validated by the canonical packager, never by module authors; - targets: `any`, or -[-] (linux-x64-glibc for Signage); - ekklesia.package.json (packageFormat 1): id, version, target, runtime needs, provenance, incorporated SDK versions, a `transitional` Drizzle requirement explicitly not the Module Database ABI, and per-file size and SHA-256; - archive safety: no links of any kind, no absolute or `..` paths, no special files or setuid/setgid; extraction normalizes ownership and modes; - the intended installation lifecycle and its rollback limit: code-only rollback only before new migrations apply, a backup restore after; - integrity, not authenticity; signing and trust roots deferred; - pnpm deploy explicitly not the packaging mechanism. Status Proposed, following the repository's usual lifecycle. ADR 0047 gains a cross-reference to consume canonical packages. The module development guide no longer recommends running pnpm deploy in a workspace, which corrupts that workspace's install state. No runtime, tooling, Docker, CI, Signage, or instance changes. Co-Authored-By: Claude Opus 5.5 --- .../decisions/0047-module-ci-harness.md | 2 + .../decisions/0049-module-package-v1.md | 426 ++++++++++++++++++ docs/architecture/decisions/README.md | 1 + docs/module-development/README.md | 15 +- 4 files changed, 437 insertions(+), 7 deletions(-) create mode 100644 docs/architecture/decisions/0049-module-package-v1.md diff --git a/docs/architecture/decisions/0047-module-ci-harness.md b/docs/architecture/decisions/0047-module-ci-harness.md index 9eeb0bb..72b9c2b 100644 --- a/docs/architecture/decisions/0047-module-ci-harness.md +++ b/docs/architecture/decisions/0047-module-ci-harness.md @@ -17,6 +17,8 @@ steps 1 and 3 below build the module into a deployment image, which ADR 0048 rules out. Before this proposal is accepted, the harness must run the module from module storage in the generic image. +- Should consume: [ADR 0049](0049-module-package-v1.md), when accepted — the harness installs the + module's canonical package, the artifact a church would install, rather than a workspace build. ## Context diff --git a/docs/architecture/decisions/0049-module-package-v1.md b/docs/architecture/decisions/0049-module-package-v1.md new file mode 100644 index 0000000..c15005c --- /dev/null +++ b/docs/architecture/decisions/0049-module-package-v1.md @@ -0,0 +1,426 @@ +# 0049: A module release is one immutable, self-contained package: Ekklesia Module Package v1 + +- Status: Proposed +- Date: 2026-10-07 +- Supersedes: none +- Extends: [0048](0048-generic-image-and-installed-module-storage.md) +- Related: [ADR 0006](0006-versioned-capabilities-and-compiled-module-runtime.md), + [ADR 0007](0007-module-owned-drizzle-schemas.md), + [ADR 0008](0008-module-owned-migration-contracts.md), + [ADR 0009](0009-transactional-module-migrations.md), + [ADR 0016](0016-module-database-access.md), + [ADR 0021](0021-modules-as-hostable-products.md), + [ADR 0023](0023-module-ui-build-pipeline.md), + [ADR 0028](0028-module-operational-lifecycle.md), + [ADR 0040](0040-module-stylesheets-and-artifact-isolation.md), + [ADR 0041](0041-module-browser-contributions.md), + [ADR 0044](0044-self-hosted-reference-deployment.md), + [ADR 0047](0047-module-ci-harness.md) + +## Context + +ADR 0048 took modules out of the platform image. An installation keeps them in its own module +storage, mounted read-only, and discovers them at a controlled restart. Each installed module must +be a self-contained directory: the platform lends it no dependencies. ADR 0048 deferred the format +a module is delivered in. + +That gap is now the obstacle. Nothing yet produces an installed module directory except a test-only +Dockerfile, so a module release cannot be built, distributed, and installed on its own. An +investigation against the module template and Signage 0.1.2 established: + +- **`dist/` alone does not run.** Compiled server and schema code imports `@ekklesia/module-sdk` + (and through it `@ekklesia/contracts`, zod, and semver) and `drizzle-orm` by name, and Node never + resolves those from the platform's `node_modules`. +- **Bundling works.** Bundling the template's server and schema with esbuild gave about 720 KB of + backend code and no `node_modules` at all. Inside the generic image, run as its unprivileged + user from a read-only mount at `/var/lib/ekklesia/modules`, it passed module, schema, and + migration validation, and its server entrypoint imported. Core's own Drizzle instance built a + query over the bundled module's table. +- **Native dependencies cannot be bundled.** Signage needs `sharp`, a Node-API addon whose + platform packages carry an 18.6 MB `libvips` shared library for one operating system, CPU, and + C library (Linux, x64, glibc ≥ 2.28). With its backend bundled and only `sharp` and its own + dependencies kept beside it, Signage was 22 MB and contained no link. It validated and processed + an image inside the same image. +- **`pnpm deploy` is the wrong tool.** Non-legacy mode refuses unless the whole workspace switches + to injected workspace packages. Legacy mode leaves a link escaping the package. Every mode + ships packages nothing loads at run time (React and its DOM, type packages, database drivers + pulled in as peers) and leaves `workspace:` ranges in copied manifests. Running it inside a + workspace rewrites that workspace's install state to production-only, which then breaks the + workspace's own builds. It did so in the investigation's scratch workspaces, and earlier in a + developer's own checkout. +- **A deterministic archive is cheap.** A tar archive with sorted paths, fixed owner and + timestamps, and a gzip header without a name or time produced the same SHA-256 twice. Gzip packed + Signage to 8.5 MB. + +Module releases also need a contract that does not depend on where they are hosted. Ekklesia's +source and CI live on a Gitea server today; a marketplace, another host, object storage, or offline +media may carry the same release tomorrow. + +## Decision + +### Purpose + +A separately versioned Ekklesia module is distributed as an **Ekklesia Module Package**: one +immutable, self-contained archive. Installing, updating, or removing a module never rebuilds the +platform image (ADR 0048). + +The package contract is independent of marketplace discovery, of the source-code host, of any +artifact registry, of the church installing it, and of the URL it was fetched from. Catalogues, +marketplaces, and signing are separate concerns, and none of them is decided here. + +### The artifact is a deterministic `.tar.gz` + +Module Package v1 is a gzip-compressed tar archive. Its identity is its contents, never its +transport: the same archive may travel through Gitea, GitHub, an OCI registry, object storage, +offline media, or a future Ekklesia marketplace without any change to what it installs. No registry, +host, or OCI concept is part of the package format. + +The canonical packager produces the archive **deterministically**: the same packager version and +the same inputs produce byte-identical output. At minimum: + +- **Entries.** Every directory and regular file appears in byte-wise sorted path order, directories + before their contents. +- **Ownership.** uid and gid are 0 and the user and group names are empty. +- **Timestamps.** Every entry's modification time is 0, and the archive holds no access, change, or + creation time. +- **Modes.** Directories are `0755` and files `0644` (see [Archive safety](#archive-safety)). +- **No host metadata.** No extended attributes, ACLs, or archive-level headers that carry host or + time information. +- **gzip.** The header has no file name and a modification time of 0, at a fixed compression level. + +The SHA-256 of the complete archive is distributed beside it, as a separate `.sha256` file in the +`sha256sum` format. The **recommended** file name is `--.tar.gz`, +for example `signage-0.1.2-linux-x64-glibc.tar.gz`. A file name is a convenience, never trusted: +what the archive is comes from its contents and its digest. + +### What a package contains + +A v1 archive contains exactly one top-level directory, named with the module id. Its contents are +the installed module directory ADR 0048 requires: + +```text +/ + package.json generated runtime metadata + ekklesia.module.json the module's manifest, unchanged + ekklesia.package.json package metadata (below) + THIRD_PARTY_NOTICES licence and notice material for everything bundled or included + dist/ bundled backend entrypoints, their shared chunks, and the declared + UI, display, and contribution artifacts + drizzle/ migration SQL, journal, snapshots, and _ekklesia.json, when declared + node_modules/ only when the package needs a native runtime dependency (below) +``` + +Every entrypoint the manifest declares exists at its declared path. + +It contains nothing else, and in particular: + +- source files, unless something at run time reads them; +- tests; +- TypeScript declarations; +- source maps, until a later decision permits them; +- development dependencies; +- any `workspace:` reference; +- any link (see [Archive safety](#archive-safety)). + +**The runtime `package.json` is generated by the packager, never copied** from the module's +development workspace. It holds only what module discovery reads: +- `name` and `version` (equal to the manifest's version); +- `"private": true`; +- `"type": "module"`; +- `"ekklesia": { "layer": "module" }`. + +It declares no dependencies, scripts, exports, or imports, because nothing at run time resolves +through them. + +The manifest is copied unchanged. It stays the module's authoritative runtime contract, and its +`compatibility.platform` and `compatibility.moduleSdk` ranges stay the only place those are stated. + +### Backend code is bundled + +The canonical packager bundles the module's backend entrypoints (server, schema, and jobs) with +every pure-JavaScript dependency they use. That includes `@ekklesia/module-sdk`, +`@ekklesia/contracts`, and ordinary libraries. The bundles are ES modules at the manifest's declared +paths, and they may share chunks under `dist/`. Browser artifacts are already self-contained bundles +(ADR 0023, ADR 0040, ADR 0041) and are packaged as built. + +The platform lends an installed module nothing (ADR 0048). **`node_modules` in a package is the +exception, not the default.** A dependency stays outside the bundle only when it must exist as +separate files at run time: a native Node addon, or a package that is equally platform-specific. +Then the package carries it, and exactly its own runtime dependencies, flat under +`node_modules/`. + +**The canonical packager decides what stays external, and validates it.** It refuses to package a +module whose bundled code still needs a file it did not include, or whose externals are not native +or platform-specific. Choosing which dependencies to leave unbundled is not part of the module +author's interface. + +### Targets + +Every package has one **target**: + +- **`any`**: the package contains no native or platform-specific file. It runs wherever its declared + Node.js range is met. +- **`-[-]`**: the package contains native code for exactly one environment. + - `` and `` use Node.js's own `process.platform` and `process.arch` names. + - On Linux, `` is required and is `glibc` or `musl`. On other platforms it is omitted. + +Examples: `linux-x64-glibc`, `linux-arm64-glibc`, `linux-x64-musl`. Signage, because of `sharp`, is +`linux-x64-glibc` for the current production image. + +A package has exactly one target. A module that ships for several environments publishes one +package per target. Packages carrying several architectures at once are not part of v1. + +### `ekklesia.package.json` + +For example (the values are illustrative): + +```json +{ + "packageFormat": 1, + "module": { "id": "signage", "version": "0.1.2" }, + "target": "linux-x64-glibc", + "runtime": { + "node": ">=24.11.0 <25", + "nodeApi": 9, + "libcVersion": ">=2.28" + }, + "provenance": { + "revision": "86ae907f59458508e3277666cdd214f5bd7c1078", + "repository": "https://example.org/your-org/signage.git", + "packager": { "name": "@ekklesia/module-pack", "version": "0.2.0" } + }, + "incorporates": { + "@ekklesia/module-sdk": "0.1.2", + "@ekklesia/contracts": "0.1.2" + }, + "transitional": { + "databaseRuntime": { "drizzle-orm": "0.45.2" } + }, + "files": [ + { "path": "THIRD_PARTY_NOTICES", "size": 48211, "sha256": "…" }, + { "path": "dist/server/index.js", "size": 1346, "sha256": "…" } + ] +} +``` + +The fields are: + +- **`packageFormat`**: the integer `1`. An installer refuses a format it does not implement. Format + versions are strict cutovers, like manifest versions. +- **`module.id`, `module.version`**: must equal the manifest's and the runtime `package.json`'s. + The top-level directory must be named with the id. +- **`target`**: as defined above. +- **`runtime`**: what the package needs from the Node.js runtime that will load it. + - `node`, a semantic-version range, is always present. + - A native package also states: + - `nodeApi`: the minimum Node-API version, for Node-API addons; + - or `nodeAbi`: the exact `process.versions.modules`, for an addon built against one Node + ABI; + - and on Linux, `libcVersion`: the minimum C library version. +- **`provenance`**: how the package was built. + - `revision`: the exact source revision, always present. + - `repository`: informational only. Nothing trusts it, and it ties the package to no host. + - `packager`: the packager's name and version. +- **`incorporates`**: the exact versions of the Ekklesia SDK and runtime libraries bundled into the + package. The packager refuses a package whose incorporated `@ekklesia/module-sdk` does not satisfy + the manifest's own `compatibility.moduleSdk` range, so the two can never disagree. +- **`transitional`**: compatibility facts that today's implementation needs and that are not part + of the long-term contract. See [Database runtime compatibility](#database-runtime-compatibility). + An installer must understand every key it finds there or refuse the package. +- **`files`**: one entry for every regular file in the module directory **except + `ekklesia.package.json` itself**, sorted by path. Each entry has its path relative to the module + directory, its size in bytes, and its SHA-256. The set is exact: a payload file missing from the + list, or a listed file missing from the payload, invalidates the package. + +The archive's `.sha256` covers the whole archive, `ekklesia.package.json` included. The per-file +digests let an installer, or an operator later, check an extracted directory without the archive. + +Platform and Module SDK compatibility stay in `ekklesia.module.json` and are not repeated here. + +### Database runtime compatibility + +Core runs a module's queries through the platform's Drizzle instance over tables the module defined +with its own copy of Drizzle (ADR 0016, ADR 0048). Those objects only interoperate when the two +copies are interchangeable. So while it is needed, a package bundles Drizzle with its backend code, +records the version in `transitional.databaseRuntime`, and an installer refuses a package whose +Drizzle version differs from the platform's. + +**This is an implementation constraint of today's platform, not the Ekklesia Module Database ABI.** +Object identity and version agreement between two Drizzle copies are not a contract this ADR +establishes or that module authors may rely on. A separate Module Database ABI decision will replace +it, and that ABI will make the platform's ORM an implementation detail. When it lands, the +`transitional.databaseRuntime` requirement goes away under the format-versioning rules here. + +### Licence obligations + +Bundling copies third-party code into the module's own files, where the licence headers that came +with it may not survive. **Every package therefore carries `THIRD_PARTY_NOTICES`**: a generated +plain-text file at the module directory's root. It names each third-party package bundled into +or included in the package, with its version and licence, and holds the licence and notice text +that redistribution requires. + +The packager's implementation must establish exactly how licence and notice material is collected +and preserved, including licence comments that bundling would otherwise strip. Until it does, no +package is treated as a distributable release. + +### Archive safety + +**v1 packages contain no links of any kind.** Neither symbolic nor hard links are permitted, so no +link can escape the package, and there is no case where one is allowed. Native dependencies were +packaged without any. + +A conforming archive also contains no: + +- absolute path; +- `..` path segment; +- path outside the single top-level module directory; +- device node, FIFO, or socket; +- setuid, setgid, or sticky bit. + +It holds only directories and regular files. + +**Extraction normalizes ownership and modes.** Everything is owned by the installation, not by the +uid or gid in the archive. Directories are `0755` and files `0644`, readable by the API's +unprivileged user and writable by nobody but the installer. No file in a v1 package is executable. +Loading JavaScript and native addons needs no execute bit. + +An installer refuses an archive that breaks any of these rules, and never extracts it partially into +module storage. + +### Installation model + +This ADR records the intended installation lifecycle; it does not implement it. + +```text +acquire the archive +→ verify the archive's SHA-256 +→ extract into a hidden staging directory inside module storage +→ validate ekklesia.package.json, then every file's size and SHA-256 +→ validate the target and runtime against this platform and host +→ validate the manifest and the complete candidate inventory: the installed modules, with this + candidate in place of any version of the same module +→ load the backend entrypoints once in an isolated, short-lived validation process +→ produce the migration plan +→ stop the API +→ swap the candidate into the active module directory, keeping the previous version +→ apply migrations, only with explicit operator approval +→ start the API +→ verify readiness +``` + +Details: + +- **Hidden directories.** Staging and previous versions live in dot-prefixed directories inside + module storage. Module discovery ignores hidden entries (ADR 0048), so neither is ever loaded. +- **The swap.** It happens only while the API is stopped. It is as close to atomic as the + filesystem allows, and an interrupted swap is recoverable from the kept previous version. +- **Who writes.** Module storage stays read-only to Ekklesia. Whatever performs the installation + writes it from outside the API process (ADR 0048). +- **No hot loading.** Discovery still happens only when a process starts. +- **Migrations stay explicit.** Nothing in this lifecycle applies a migration on its own (ADR 0009, + ADR 0044). + +### Rollback + +- **Before any migration from the new version is applied,** restoring the previous module directory + and restarting is a supported rollback. +- **After forward migrations are applied,** restoring older code is **not** a supported rollback. + The older package lacks the newer migrations, so migration planning refuses it as a downgrade + (ADR 0048). Even running it would mean old code against a newer schema. A true rollback then + means restoring the matching database and storage backup, unless a future, module-specific + compatibility mechanism says otherwise. + +There are no down migrations (ADR 0008, ADR 0009). An installer must make the point of no return +explicit: back up before applying, and offer code-only rollback only while no new migration has been +applied. + +### Trust + +The SHA-256 digest proves integrity: the archive is the one whose digest you were given. It says +nothing about who published it. Package signatures, publisher trust roots, marketplace verification, +and trust levels for third-party modules are deferred. + +Installed modules remain trusted backend code under ADR 0048. A conforming package is not sandboxed, +and passing validation does not make a module safe; it makes it well-formed. + +### Canonical tooling + +Packages are produced by one Ekklesia-owned packager, proposed as the workspace package +`packages/module-pack` with the command `ekklesia-pack-module`, run in a module's CI. Its command +line is an implementation detail; this ADR defines what it must produce. + +**`pnpm deploy` is not the packaging mechanism**, for the reasons in Context: +- it includes runtime dependencies nothing loads; +- it leaves `workspace:` metadata; +- it leaves links that escape the package, or needs a workspace-wide linking change to avoid them; +- it rewrites the install state of the workspace it runs in. + +The repository's test-only packaging of the module template +(`scripts/lib/test-module-package.Dockerfile`) is replaced by the canonical packager when one +exists. + +### Relationship to the module CI harness + +ADR 0047, still proposed, is already marked for revision against ADR 0048. Its harness should +consume canonical module packages: a module's CI builds its package, and the harness installs that +exact archive into module storage of the generic image. The harness then tests the artifact a church +would install, not a workspace build of it. Until the packager exists, the harness and smoke tests +may keep their current test-only packaging. + +## Consequences + +- A module release becomes one file plus its digest, built once by the module's CI and installable + on any compatible Ekklesia installation without touching the platform image. +- Module authors gain a packaging step and a target per native environment. A pure-JavaScript + module ships one `any` package; Signage ships `linux-x64-glibc` packages for the current fleet. +- Packages are small and auditable: bundled code plus, only where needed, a native dependency + tree. There are no links and no development dependencies, and every file is listed with its + digest. +- Each package carries its own copy of the SDK and contracts. Version agreement with the platform is + checked when the package is built, against the manifest's `moduleSdk` range, and Core keeps + validating module definitions structurally (ADR 0048). +- Rolling back after a schema-changing update is a backup restore. Operators must take a backup + before approving migrations. +- Licence handling becomes the packager's responsibility, and until it is proven no package + counts as a release. +- The Drizzle requirement is visible in package metadata as `transitional`, so its eventual removal + is a recorded format change and not a silent behaviour change. + +## Alternatives + +- **`pnpm deploy` as the packager.** Rejected for the reasons in Context. +- **Copying the workspace's dependency tree without bundling.** Rejected as the default. It ships + far more than runs (49 MB against 22 MB for Signage), keeps `workspace:` ranges, and makes every + pure-JavaScript dependency a file-layout concern. Bundling is already how browser artifacts are + built. +- **Platform-lent runtime dependencies.** Rejected by ADR 0048. +- **Installing the SDK from a registry when the module is installed.** Rejected: installation would + need network access and a resolver on every installation, and an installation's code would depend + on what a registry served that day. Publishing the SDK packages for module *builds* is a separate, + compatible future step. +- **Module-author-chosen externals.** Rejected: an arbitrary external is an undeclared runtime + dependency the platform cannot verify, and it reintroduces the `node_modules` layout as a + contract. +- **An OCI artifact as the package format.** Not adopted for v1: it needs registry tooling and login + to install, and adds nothing to integrity that the digest does not already give. Nothing here + prevents carrying the same `.tar.gz` inside an OCI artifact. +- **`.tar.zst`.** Not adopted for v1: about 17 % smaller for Signage at a high compression level, + but about three times slower to produce there, and less universally available than gzip. A later + format version may add it. +- **Packages carrying several architectures at once.** Not adopted for v1: each extra architecture + adds about 18.5 MB for Signage, for a fleet that runs one architecture. A later format version may + allow it. +- **Permitting internal symbolic links.** Rejected: links are where escape checks are subtle, and + the native dependencies that motivated the question were packaged without any. + +## Deferred + +- Package signatures and publisher trust roots. +- A marketplace or catalogue, and registry or hosting-provider integration. +- An installer, and any installer UI; an automatic update service. +- The purge workflow (ADR 0048). +- Packages carrying several architectures at once. +- The stable Module Database ABI that retires `transitional.databaseRuntime`. +- Source maps and other debugging material in or beside packages. +- Publishing the Ekklesia SDK packages for module builds outside a platform workspace. +- Moving Signage to the new package format. diff --git a/docs/architecture/decisions/README.md b/docs/architecture/decisions/README.md index de801d2..b7ac19d 100644 --- a/docs/architecture/decisions/README.md +++ b/docs/architecture/decisions/README.md @@ -57,3 +57,4 @@ ADRs capture durable choices that affect more than one package or phase. - [0046: Ekklesia's branding is never forced on an installation](0046-installation-owned-branding.md) — Proposed - [0047: Every module proves itself inside Ekklesia, with one harness the platform publishes](0047-module-ci-harness.md) — Proposed - [0048: Ekklesia ships one generic image, and installed modules live outside it](0048-generic-image-and-installed-module-storage.md) — Accepted +- [0049: A module release is one immutable, self-contained package: Ekklesia Module Package v1](0049-module-package-v1.md) — Proposed diff --git a/docs/module-development/README.md b/docs/module-development/README.md index bd69d3e..c84fb03 100644 --- a/docs/module-development/README.md +++ b/docs/module-development/README.md @@ -106,13 +106,14 @@ a constraint of today's implementation, not a permanent contract. Browser artifacts are already self-contained bundles, so nothing changes for them. -From a workspace, `pnpm --filter deploy --prod --legacy ` produces such a -directory; remove the self-reference pnpm leaves at -`node_modules/.pnpm/node_modules/`, which points back into the workspace. The -repository's own smoke test packages the module template this way -(`scripts/lib/test-module-package.Dockerfile`). That is a test fixture, not a package format: the -format modules are delivered in, how they are signed, and how they reach an installation are not -decided yet. +The format modules are delivered in is proposed in +[ADR 0049](../architecture/decisions/0049-module-package-v1.md): one deterministic archive per +target, with bundled backend code, produced by an Ekklesia-owned packager that does not exist yet. +Until it does, the repository's smoke test packages the module template with `pnpm deploy` in a +disposable Docker build (`scripts/lib/test-module-package.Dockerfile`), a test fixture and not a +package format. Never run `pnpm deploy` in your own workspace: it rewrites that workspace's install +state to production-only, and the next build or `pnpm verify` then fails until a normal +`pnpm install` repairs it. The repository's boundary checks (`pnpm boundaries`) prove only source modules in a workspace. An installed package normally has no source, so the platform cannot check it the same way. Your