feat: Ekklesia Module Package v1, the canonical packager and verifier (ADR 0049) #66

Merged

Zayin_Admin merged 2 commits from feat/module-package-v1 into main 2026-10-07 18:33:28 +00:00

Conversation 1 Commits 2 Files Changed 47

+5083 -151

ekklesia-bot commented 2026-10-07 17:46:07 +00:00

Member

Copy Link

Copy Source

Implements the first slice of ADR 0049, Ekklesia Module Package v1: the canonical packager, Core's package verifier, the portable any target only, the module template as the end-to-end proof, and the split deploy smoke consuming the canonical package. ADR 0049 is accepted by a status-line change, following the 0044 precedent.

Not included: Signage packaging, native targets (linux-x64-glibc), marketplace, registry publishing, signing, installer UI, automatic updates, purge, or any Module Database ABI change. rog-ekklesia-instance and Ekklesia_Signage are untouched.

What a package is

example-0.1.0-any.tar.gz        + example-0.1.0-any.tar.gz.sha256  ("<hex>  <file name>")
└── example/
    ├── package.json            generated: name, version, private, "type": "module", ekklesia.layer
    ├── ekklesia.module.json    the manifest, byte for byte
    ├── ekklesia.package.json   packageFormat 1, identity, target, runtime, provenance,
    │                           incorporates, transitional.databaseRuntime, files[path,size,sha256]
    ├── THIRD_PARTY_NOTICES
    ├── dist/server/index.js    bundled backend entrypoints
    ├── dist/schema/index.js
    ├── dist/chunks/chunk-*.js  shared backend code
    ├── dist/ui/ dist/display/ dist/contributions/   browser artifacts, exactly as built
    └── drizzle/                migrations

• Format. Deterministic ustar and gzip: sorted paths with directories first, uid/gid 0, empty names, mtime 0, 0755/0644, no links, no extended headers, two end blocks; gzip at a fixed level with no name or time.
• Determinism, proven. A clean worktree, a developer checkout, and the clean Docker build (Node 24.18 at /opt/source) produced byte-identical archives for commit 35ef31d: 91e5dfee80880027bd1c2e1796513f07023c03847e9ca3870b51f8272c3accf7, 426,773 bytes (2.56 MB unpacked, 13 payload files).

Packager — @ekklesia/module-pack (new, tooling)

ekklesia-pack-module --module <built module dir> --target any --out <dir> \
  --source-revision <full commit id> [--repository <url>]

Root alias: pnpm modules:pack ….

• Bundling. esbuild bundles the server, schema, and jobs entrypoints with every pure-JavaScript dependency, including @ekklesia/module-sdk, @ekklesia/contracts, Drizzle, zod, and semver. The only imports left are Node built-ins. Packaging fails on:
  • an esbuild warning;
  • an unresolved import;
  • an external dependency;
  • a native or platform-specific package: one with os/cpu/libc fields, gypfile or binding.gyp, a .node file, or a dependency that declares a platform (the sharp pattern).
• Browser artifacts are copied as built. The UI builder's new analyzeModuleUi must first prove each one byte-identical to a fresh in-memory build.
• Never shipped: src/, tests, .d.ts, source maps, development dependencies, or workspace: ranges.
• No pnpm deploy, nothing installed. It writes only the archive and its .sha256, never runs pnpm deploy, and never links or installs anything. The root pnpm install state was hashed before and after packing, in tests and in a clean worktree, and is unchanged.

THIRD_PARTY_NOTICES

• Discovery. Every input file of every bundle (backend metafile, plus each browser artifact's inputs, including nested stylesheet imports) is traced to its package. A package installed under node_modules is third-party. Workspace packages (@ekklesia/*) and the module itself are not; the Ekklesia packages are recorded under incorporates.
• Content. Each component gets its name, version, declared licence, where it is used (backend or browser), and the full text of every LICENSE*/LICENCE*/COPYING*/COPYRIGHT*/NOTICE* file at its root, plus any SEE LICENSE IN target. Line endings are normalized and everything is sorted.
• Duplicates. A component used by several bundles appears once, with every use listed. Different components are never merged.
• Gaps fail. A missing or UNLICENSED licence fails packaging, and so does missing licence text. The exception is a licence whose standard text is complete without a copyright line: Apache-2.0, vendored from apache.org with a SHA-256 pinned by a test, which the file then labels as supplied. drizzle-orm 0.45.2 ships no licence file, so it needs exactly this.
• Template result: drizzle-orm (Apache-2.0), react, react-dom, scheduler, zod (MIT), and semver (ISC). A test checks this against every node_modules path esbuild names inside the shipped JavaScript.

Verifier — Core

verifyModulePackage (in @ekklesia/core/modules) and the command:

node core/dist/cli/verify-module-package.js <archive> [--sha256 <hex> | --sha256-file <file>] [--extract-to <new dir>]

Root alias: pnpm modules:verify-package …. In order, it:

1. checks the archive digest;
2. reads the archive in memory with strict ustar parsing and size limits, refusing absolute or .. paths, symbolic and hard links, devices, FIFOs and other types, setuid/setgid/sticky modes, a second top-level directory, and duplicate or conflicting entries;
3. checks the metadata format, including unknown transitional keys;
4. checks the strict schema and the exact file set with every size and SHA-256;
5. checks the generated package.json shape, and id and version agreement across metadata, package.json, manifest, and directory name;
6. checks the target (any only), the Node.js range, the incorporated SDK against the manifest's moduleSdk range, and the transitional Drizzle agreement with Core's own Drizzle version;
7. writes the package to a scratch directory and runs the platform's module and migration discovery;
8. loads the backend entrypoints in a forked process with an empty environment, reporting over IPC;
9. runs schema discovery;
10. only then writes the verified tree into a directory that must not exist yet, with normalized modes.

Errors carry fixed messages and no paths, entry names, or module output.

Other changes

• @ekklesia/contracts: the ekklesia.package.json schema, target grammar, and runtime package.json builder.
• @ekklesia/module-ui-build: analyzeModuleUi. It also fixes a latent determinism bug: the nested stylesheet build now runs from the module directory, so artifacts no longer depend on the caller's working directory. That was found because the packager's freshness check failed.
• turbo.json: module-pack's tests run after the template's own tests, which rebuild its UI in place.
• Smoke. scripts/lib/build-module-package.Dockerfile replaces the test-only pnpm deploy packaging and only provides a build environment for the canonical packager.

Smoke result (split)

One generic image is built once and used throughout:

1. it boots with no modules;
2. the template package is built by the canonical packager;
3. the same image verifies it, running unprivileged with a read-only root filesystem, and extracts it to staging;
4. the verified tree is installed into module storage;
5. migrations are planned and explicitly applied, and the API restarts;
6. module HTTP, UI, display, and browser checks pass;
7. the module is removed and the API restarts: Core is healthy, planning reports [retained], and the data survives;
8. the exact same archive is verified and installed again: the history is strict and valid, and the data is back;
9. Doctor reports READY.

No second platform image is built.

Verification

Gate	Result
pnpm verify	passed: contracts 232, core 1059, api 510, module-pack 39, module-ui-build 40, template 20, shell 219, root 91, all others green; boundaries (14 packages), module and migration validation, identity schema, build
pnpm test:migrations:integration	passed, 14 files and 308 tests
pnpm deploy:config	valid
bash scripts/check-image.sh	passed: no modules, no template, identical runtime file system with a populated modules/ (17,504 paths, identical layers)
same-origin deploy smoke	passed
split deploy smoke	passed, as above
clean-worktree determinism	three byte-identical archives, 91e5dfee…; root pnpm install state unchanged by packaging

Notes for review

• Ekklesia declares no licence. There is no LICENSE file and no package declares one. The packager treats @ekklesia/* workspace packages as the platform's own code and records them in incorporates, not in the notices. Before third parties redistribute packages that bundle the SDK, the project needs to declare its licence.
• UI source-map comments. The UI artifacts keep their trailing //# sourceMappingURL=…map comments, because they ship byte for byte as built while the maps themselves are excluded. They are harmless dangling references.
• Isolated verification. The verifier validates a candidate on its own. A module with required module dependencies would fail; validating it within the installed inventory belongs to the installer slice.
• Known flake. The pre-existing 5-second Doctor import-safety test timed out once under full parallel load. It passed on rerun and alone, and on the base commit.

🤖 Generated with Claude Code

Implements the first slice of **ADR 0049, Ekklesia Module Package v1**: the canonical packager, Core's package verifier, the portable `any` target only, the module template as the end-to-end proof, and the split deploy smoke consuming the canonical package. **ADR 0049 is accepted** by a status-line change, following the 0044 precedent. Not included: Signage packaging, native targets (`linux-x64-glibc`), marketplace, registry publishing, signing, installer UI, automatic updates, purge, or any Module Database ABI change. `rog-ekklesia-instance` and `Ekklesia_Signage` are untouched. ## What a package is ``` example-0.1.0-any.tar.gz + example-0.1.0-any.tar.gz.sha256 ("<hex> <file name>") └── example/ ├── package.json generated: name, version, private, "type": "module", ekklesia.layer ├── ekklesia.module.json the manifest, byte for byte ├── ekklesia.package.json packageFormat 1, identity, target, runtime, provenance, │ incorporates, transitional.databaseRuntime, files[path,size,sha256] ├── THIRD_PARTY_NOTICES ├── dist/server/index.js bundled backend entrypoints ├── dist/schema/index.js ├── dist/chunks/chunk-*.js shared backend code ├── dist/ui/ dist/display/ dist/contributions/ browser artifacts, exactly as built └── drizzle/ migrations ``` - **Format.** Deterministic ustar and gzip: sorted paths with directories first, uid/gid 0, empty names, mtime 0, `0755`/`0644`, no links, no extended headers, two end blocks; gzip at a fixed level with no name or time. - **Determinism, proven.** A clean worktree, a developer checkout, and the clean Docker build (Node 24.18 at `/opt/source`) produced **byte-identical** archives for commit `35ef31d`: `91e5dfee80880027bd1c2e1796513f07023c03847e9ca3870b51f8272c3accf7`, **426,773 bytes** (2.56 MB unpacked, 13 payload files). ## Packager — `@ekklesia/module-pack` (new, tooling) ``` ekklesia-pack-module --module <built module dir> --target any --out <dir> \ --source-revision <full commit id> [--repository <url>] ``` Root alias: `pnpm modules:pack …`. - **Bundling.** esbuild bundles the server, schema, and jobs entrypoints with every pure-JavaScript dependency, including `@ekklesia/module-sdk`, `@ekklesia/contracts`, Drizzle, zod, and semver. The only imports left are Node built-ins. Packaging fails on: - an esbuild warning; - an unresolved import; - an external dependency; - a native or platform-specific package: one with `os`/`cpu`/`libc` fields, `gypfile` or `binding.gyp`, a `.node` file, or a dependency that declares a platform (the `sharp` pattern). - **Browser artifacts** are copied as built. The UI builder's new `analyzeModuleUi` must first prove each one byte-identical to a fresh in-memory build. - **Never shipped:** `src/`, tests, `.d.ts`, source maps, development dependencies, or `workspace:` ranges. - **No pnpm deploy, nothing installed.** It writes only the archive and its `.sha256`, never runs `pnpm deploy`, and never links or installs anything. The root pnpm install state was hashed before and after packing, in tests and in a clean worktree, and is unchanged. ### THIRD_PARTY_NOTICES - **Discovery.** Every input file of every bundle (backend metafile, plus each browser artifact's inputs, including nested stylesheet imports) is traced to its package. A package installed under `node_modules` is third-party. Workspace packages (`@ekklesia/*`) and the module itself are not; the Ekklesia packages are recorded under `incorporates`. - **Content.** Each component gets its name, version, declared licence, where it is used (backend or browser), and the full text of every `LICENSE*`/`LICENCE*`/`COPYING*`/`COPYRIGHT*`/`NOTICE*` file at its root, plus any `SEE LICENSE IN` target. Line endings are normalized and everything is sorted. - **Duplicates.** A component used by several bundles appears once, with every use listed. Different components are never merged. - **Gaps fail.** A missing or `UNLICENSED` licence fails packaging, and so does missing licence text. The exception is a licence whose standard text is complete without a copyright line: Apache-2.0, vendored from apache.org with a SHA-256 pinned by a test, which the file then labels as supplied. **`drizzle-orm` 0.45.2 ships no licence file**, so it needs exactly this. - **Template result:** drizzle-orm (Apache-2.0), react, react-dom, scheduler, zod (MIT), and semver (ISC). A test checks this against every `node_modules` path esbuild names inside the shipped JavaScript. ## Verifier — Core `verifyModulePackage` (in `@ekklesia/core/modules`) and the command: ``` node core/dist/cli/verify-module-package.js <archive> [--sha256 <hex> | --sha256-file <file>] [--extract-to <new dir>] ``` Root alias: `pnpm modules:verify-package …`. In order, it: 1. checks the archive digest; 2. reads the archive in memory with strict ustar parsing and size limits, refusing absolute or `..` paths, symbolic and hard links, devices, FIFOs and other types, setuid/setgid/sticky modes, a second top-level directory, and duplicate or conflicting entries; 3. checks the metadata format, including unknown `transitional` keys; 4. checks the strict schema and the exact file set with every size and SHA-256; 5. checks the generated `package.json` shape, and id and version agreement across metadata, `package.json`, manifest, and directory name; 6. checks the target (`any` only), the Node.js range, the incorporated SDK against the manifest's `moduleSdk` range, and the transitional Drizzle agreement with Core's own Drizzle version; 7. writes the package to a scratch directory and runs the platform's module and migration discovery; 8. **loads the backend entrypoints in a forked process with an empty environment**, reporting over IPC; 9. runs schema discovery; 10. only then writes the verified tree into a directory that must not exist yet, with normalized modes. Errors carry fixed messages and no paths, entry names, or module output. ## Other changes - **`@ekklesia/contracts`:** the `ekklesia.package.json` schema, target grammar, and runtime `package.json` builder. - **`@ekklesia/module-ui-build`:** `analyzeModuleUi`. It also fixes a latent determinism bug: the nested stylesheet build now runs from the module directory, so artifacts no longer depend on the caller's working directory. That was found because the packager's freshness check failed. - **`turbo.json`:** module-pack's tests run after the template's own tests, which rebuild its UI in place. - **Smoke.** `scripts/lib/build-module-package.Dockerfile` replaces the test-only `pnpm deploy` packaging and only provides a build environment for the canonical packager. ## Smoke result (split) One generic image is built **once** and used throughout: 1. it boots with no modules; 2. the template package is built by the canonical packager; 3. the **same image** verifies it, running unprivileged with a read-only root filesystem, and extracts it to staging; 4. the verified tree is installed into module storage; 5. migrations are planned and explicitly applied, and the API restarts; 6. module HTTP, UI, display, and browser checks pass; 7. the module is removed and the API restarts: Core is healthy, planning reports `[retained]`, and the data survives; 8. **the exact same archive** is verified and installed again: the history is strict and valid, and the data is back; 9. Doctor reports READY. No second platform image is built. ## Verification | Gate | Result | |---|---| | `pnpm verify` | passed: contracts 232, core 1059, api 510, module-pack 39, module-ui-build 40, template 20, shell 219, root 91, all others green; boundaries (14 packages), module and migration validation, identity schema, build | | `pnpm test:migrations:integration` | passed, 14 files and 308 tests | | `pnpm deploy:config` | valid | | `bash scripts/check-image.sh` | passed: no modules, no template, identical runtime file system with a populated `modules/` (17,504 paths, identical layers) | | same-origin deploy smoke | passed | | split deploy smoke | passed, as above | | clean-worktree determinism | three byte-identical archives, `91e5dfee…`; root pnpm install state unchanged by packaging | ## Notes for review - **Ekklesia declares no licence.** There is no LICENSE file and no package declares one. The packager treats `@ekklesia/*` workspace packages as the platform's own code and records them in `incorporates`, not in the notices. **Before third parties redistribute packages that bundle the SDK, the project needs to declare its licence.** - **UI source-map comments.** The UI artifacts keep their trailing `//# sourceMappingURL=…map` comments, because they ship byte for byte as built while the maps themselves are excluded. They are harmless dangling references. - **Isolated verification.** The verifier validates a candidate on its own. A module with *required* module dependencies would fail; validating it within the installed inventory belongs to the installer slice. - **Known flake.** The pre-existing 5-second Doctor import-safety test timed out once under full parallel load. It passed on rerun and alone, and on the base commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

ekklesia-bot added 1 commit 2026-10-07 17:46:07 +00:00

feat: Ekklesia Module Package v1, the canonical packager and verifier (ADR 0049) ... 35ef31df9f

A separately versioned module is now distributed as one deterministic
.tar.gz and its SHA-256, produced by an Ekklesia-owned packager and checked by
Core before anything is installed. This slice implements the portable `any`
target, with the module template as the end-to-end proof. ADR 0049 is
accepted.

@ekklesia/contracts: the Module Package v1 contract. The strict
ekklesia.package.json schema (packageFormat 1, identity, target, runtime,
provenance, incorporated Ekklesia packages, strict transitional facts, an
exact byte-wise-sorted file list with size and SHA-256), target names, and the
generated runtime package.json.

@ekklesia/module-pack (new, tooling): `ekklesia-pack-module --module <dir>
--target any --out <dir> --source-revision <commit>`.
- Bundles server, schema, and jobs with esbuild, SDK, contracts, and Drizzle
  included; only Node built-ins remain imports. Any warning, unresolved
  import, external dependency, or native or platform-specific package fails.
- Copies browser artifacts as built, after proving them current, and the
  migrations; generates package.json and ekklesia.package.json; never ships
  src, tests, declarations, source maps, or workspace ranges.
- THIRD_PARTY_NOTICES: every third-party package traced from the bundles'
  own inputs, credited once with its declared licence and every licence,
  copying, copyright, and notice file it ships. A missing licence fails; a
  missing text fails unless its standard text is complete without copyright
  lines (Apache-2.0, vendored, needed by drizzle-orm).
- Deterministic ustar and gzip: sorted paths, uid/gid 0, empty names, mtime
  0, 0755/0644, no links. Byte-identical across runs and machines.
- Writes only the archive and its .sha256; never deploys, links, or installs,
  so the workspace's install state is untouched.

Core: verifyModulePackage and `verify-module-package.js <archive>
[--sha256 | --sha256-file] [--extract-to <new dir>]`. Reads the archive in
memory and refuses links, special entries, setuid/setgid/sticky modes, unsafe
or escaping paths, and a second top-level directory; checks metadata, every
file, identity, target, Node.js range, SDK, and the transitional Drizzle
agreement; runs the platform's module and migration discovery; loads the
backend once in a forked process with an empty environment; only then
extracts, with normalized modes, into a new directory. Errors are sanitized.

@ekklesia/module-ui-build: analyzeModuleUi reports each artifact's inputs
and whether the built file is current. The nested stylesheet build now works
from the module directory, so artifacts no longer depend on the caller's
working directory.

The split deploy smoke builds the package with the canonical packager
(scripts/lib/build-module-package.Dockerfile replaces the test-only pnpm
deploy packaging), verifies it with the same generic image, installs the
verified tree, removes it, and reinstalls the exact same archive.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Zayin_Admin added 1 commit 2026-10-07 18:19:34 +00:00

fix(smoke): hand the verifier the package's archive and digest as one readable release pair ... 27d85e0162

The split deploy smoke failed in CI with "The SHA-256 file could not be read."
BuildKit's local exporter creates its output directory (`--output
type=local,dest=$work/module-package`) with mode 0700, owned by whoever ran the
build. That directory was mounted into the verifier, which runs as the image's
unprivileged `node` user (uid 1000). In CI the build runs as root, so the
directory was root:root 0700 and the verifier could read neither file; the
digest file is simply read first. On a workstation the build runs as uid 1000,
the same uid as `node`, so it was readable by coincidence.

scripts/lib/module-package-release.sh turns one build's output into the release
pair: it requires exactly one archive and its .sha256 and nothing else, checks
that the digest line names that archive, then creates a fresh directory that
must not exist yet (0755) and copies both files into it (0644). The smoke stages
and mounts only that directory; BuildKit's output directory is never mounted.
Archive-digest verification stays mandatory: the verifier still reads the
.sha256 file. Nothing in the verifier changed.

Tests: the helper is driven in bash against a 0700 build output, and refuses a
missing, malformed, or mismatched digest, stray output, a second archive, and
an existing release directory, creating nothing. A static test pins the smoke
to the release pair and the verifier's single mount with --sha256-file.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Zayin_Admin approved these changes 2026-10-07 18:30:12 +00:00

Zayin_Admin merged commit 22caa1dceb into main 2026-10-07 18:33:28 +00:00

Zayin_Admin deleted branch feat/module-package-v1 2026-10-07 18:33:28 +00:00

Zayin_Admin referenced this issue from a commit 2026-10-07 18:38:35 +00:00

chore: license Ekklesia under AGPL-3.0-only

ekklesia-bot referenced this pull request 2026-10-07 18:38:51 +00:00

chore: license Ekklesia under AGPL-3.0-or-later #67

Zayin_Admin referenced this issue from a commit 2026-10-08 22:27:59 +00:00

chore(release): prepare 0.2.0

ekklesia-bot referenced this pull request 2026-10-08 22:28:17 +00:00

chore(release): prepare 0.2.0 #82

Sign in to join this conversation.

Reviewers

No Reviewers

Zayin_Admin

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

Joey Zayin_Admin ekklesia-bot

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Zayin/Ekklesia#66