feat: native linux-x64-glibc module packages and reviewed license supplements (ADR 0049, ADR 0052)
CI / verify (pull_request) Successful in 1m6s
CI / infrastructure-smoke (pull_request) Successful in 27s
CI / migration-smoke (pull_request) Successful in 1m54s
CI / image-build (pull_request) Successful in 29s
CI / deploy-smoke (same-origin) (pull_request) Successful in 43s
CI / deploy-smoke (split) (pull_request) Successful in 1m30s

Implements ADR 0049's deferred native target and ADR 0052's reviewed
license supplements. The proof case is a real module using sharp.
packageFormat stays 1, and `any` is unchanged.

Packager (@ekklesia/module-pack):
- --target linux-x64-glibc. Native or platform-specific packages, found by
  the same generic rule that refuses them in `any`, stay outside the bundle.
- Each native package's production closure is walked through its own
  dependencies, installed optionalDependencies that match the target, and
  required peers. It is copied as real files flat under node_modules/<name>,
  with no links, dev dependencies, declarations, source maps, or other
  platforms' packages. Missing or foreign-platform required dependencies,
  version conflicts, links, foreign machines, and Node.js range conflicts
  fail closed.
- runtime.nodeApi, nodeAbi, and libcVersion are read from the binaries by a
  small ELF reader: the addon's node_api_module_get_api_version_v1 and the
  newest GLIBC_ symbol version needed. No shelling out. A native target with
  no addon is refused.
- ADR 0052 supplements: package-provided material stays primary. An exact
  match on name, version, and ekklesia-content-v1 digest (sorted normalized
  paths with each file's size and SHA-256; no metadata) adds labelled
  material after the package's own. Any mismatch, a declared-licence
  conflict, or a malformed or unreviewed supplement file fails. Missing
  material with no supplement still fails.

Supplement for @img/[email protected]:
- 184 components and 143 unique texts, assembled from the exact upstream
  sources sharp-libvips v1.3.3 builds linux-x64 from: 27 C/C++ libraries,
  glib's gvdb, librsvg's Rust crates (reproduced with cargo tree), the Rust
  standard library at the rustc commit recorded in the binary, plus the
  LGPL-3.0, GPL-3.0, and MPL-2.0 texts. Each source is recorded with its
  SHA-256 in PROVENANCE.md.
- It pins digest ekklesia-content-v1:sha256:50647b0b…, which is identical
  from the installed copy and from the registry tarball.
- LGPL/MPL source-availability determinations stay open, as ADR 0052
  states. Nothing claims they are met.

Verifier (Core):
- linux-x64-glibc is accepted only on a host whose platform, arch, glibc
  family and version, and Node-API version (from Node itself) match.
- Every ELF file must be x86-64; every .node must be an ELF object.
- `any` packages now also refuse node_modules/ and ELF files.
- Contract: native metadata must state nodeApi or nodeAbi, and Linux targets
  libcVersion (ADR 0049).

Proof:
- A checked-in AGPL native fixture packs byte-identically from unrelated
  copies (8.86 MB) and verifies with Core's verifier.
- Extracted elsewhere, it runs a sharp resize and PNG round trip in an
  empty-environment process. Every resolution stays in the package, and
  only the package's addon and libvips plus host system libraries load.

Signage is untouched.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-10-07 20:16:53 -05:00
co-authored by Claude Opus 5.5
parent f53506b3f0
commit 44d385cd7f
173 changed files with 20638 additions and 53 deletions
+15 -1
View File
@@ -683,11 +683,25 @@ Module Package v1, a deterministic `.tar.gz` with its SHA-256.
third parties, and writes the archive. The source module's string SPDX expression is required and
becomes the package metadata's authoritative `license`; the packager neither infers nor downloads
license material.
- **Native target (`linux-x64-glibc`).** Native or platform-specific dependencies stay outside
the bundle. Each one is copied with exactly its production runtime closure, as real files flat
under the package's `node_modules/`, and only the platform packages that match the target are
included. The package's `runtime` records the Node-API version and minimum glibc version, both
read from the native binaries themselves.
- **License supplements.** Where a redistributed dependency omits required licence material,
reviewed supplements checked in with the packager add it
([ADR 0052](decisions/0052-reviewed-license-supplements.md)). Each supplement matches one exact
package name, version, and content digest; anything else fails closed.
- **The verifier.** Core's `verifyModulePackage` reads the archive in memory and refuses links,
special files, and escaping paths. It checks every digest and identity, the target and runtime,
runs the platform's own module, schema, and migration validation, and loads the backend once in a
short-lived process with an empty environment. Only then does it extract anything.
- **What exists today.** Only the `any` target. A manifest-v8 ABI module carries no
- **A native package's host checks.** It is accepted only on a host whose operating system, CPU,
C library family and version, and Node-API version Node.js itself reports as compatible.
- **ELF files.** Every ELF file in it must be built for the target's machine.
- **What an `any` package may not hold:** `node_modules`, addons, and ELF files.
- **What exists today.** The `any` and `linux-x64-glibc` targets. No other operating system or CPU
target exists yet. A manifest-v8 ABI module carries no
`transitional.databaseRuntime`; Drizzle may be bundled as its private implementation detail, and
Core validates plain schema metadata without comparing ORM versions. A version 8 module that
omits `database` remains on the temporary legacy facade and keeps ADR 0049's exact transitional
+25
View File
@@ -611,6 +611,31 @@ Ekklesia Module Package v1 (ADR 0049) is proven at three levels, all in `pnpm ve
`..` paths, symbolic and hard links, device nodes and FIFOs, setuid and setgid entries, a second
top-level directory, and a duplicate entry each fail with their own sanitized code. The backend
load check runs in a real child process and selects the schema contract from the manifest.
- `packages/module-pack/tests/native-target.test.ts`: the `linux-x64-glibc` target, with a real
native dependency (`sharp`). The checked-in proof module (`tests/fixtures/native-module`) is
packed, and the test proves:
- the package carries exactly sharp's discovered runtime closure, as real files;
- nothing from another platform, and no development dependency, is included;
- `runtime` matches what the binaries declare;
- the reviewed libvips supplement is merged and labelled;
- two packs from unrelated copies are byte-identical.
Core's own verifier then accepts the package, and the extracted package runs a real image
operation in a separate process with an empty environment. Node's resolution hooks and the
process's memory map show that nothing resolves or loads from outside the package except the
host's system libraries.
- `packages/module-pack/tests/native-closure.test.ts` breaks the closure rules one at a time, with
fake native packages built around the real sharp addon's bytes:
- which platform packages are kept and which are left out;
- missing and foreign-platform dependencies;
- version conflicts, links, a foreign machine, and Node.js range conflicts;
- an unneeded native target.
- `packages/module-pack/tests/supplements.test.ts` proves ADR 0052's mechanism:
- the documented content-digest algorithm;
- that the loader refuses anything unreviewed;
- that supplement material is merged after the package's own and labelled;
- that a supplement for another version, digest, or licence fails closed;
- that the shipped libvips supplement is intact and pins the installed package exactly.
- `packages/module-pack/tests/`: the packager against generated fixture modules (payload contents,
generated `package.json`, metadata, bundled backend with only built-in imports, refusal of native
and platform-specific dependencies and of unknown or textless third-party licences, explicit
+37 -5
View File
@@ -101,7 +101,7 @@ pnpm exec ekklesia-pack-module --module . --target any --out ../packages \
--source-revision "$(git rev-parse HEAD)"
```
It writes `<id>-<version>-any.tar.gz` and its `.sha256` line, and nothing else anywhere: it never
It writes `<id>-<version>-<target>.tar.gz` and its `.sha256` line, and nothing else anywhere: it never
installs, links, or deploys, so your workspace is left exactly as it was. Never run `pnpm deploy` in
a workspace instead: it rewrites that workspace's install state to production-only.
@@ -118,7 +118,8 @@ The archive holds one directory named with your module id:
├── dist/schema/index.js
├── dist/chunks/ code your backend entrypoints share
├── dist/ui/ … dist/display/ … dist/contributions/ your browser artifacts, exactly as built
└── drizzle/ your migrations
├── drizzle/ your migrations
└── node_modules/ a native target only: native runtime dependencies, as real files
```
- **Bundled backend.** Your server, schema, and jobs entrypoints are bundled with every
@@ -129,9 +130,28 @@ The archive holds one directory named with your module id:
your development `package.json`, any `workspace:` range, and links of any kind.
- **Browser artifacts must be current.** The packager rebuilds them in memory and refuses an artifact
on disk that differs, so run your build first.
- **`any` is the only target today.** A dependency that is native or platform-specific (a compiled
addon, a package that declares an operating system or CPU, or one that loads a per-platform binary
such as `sharp`) cannot be in an `any` package, and packaging fails. Native targets come later.
- **Targets.**
- **`any`** is for pure-JavaScript modules. A dependency that is native or platform-specific (a
compiled addon, a package that declares an operating system or CPU, or one that loads a
per-platform binary such as `sharp`) cannot be in an `any` package, and packaging fails.
- **`linux-x64-glibc`** is for modules with such a dependency, on Linux, x86-64, and glibc. Pack
with `--target linux-x64-glibc` on a matching host, after installing your dependencies.
- **What a `linux-x64-glibc` package contains.**
- **Native packages stay outside the bundle.** Your ordinary JavaScript dependencies are bundled
as before. Each native package is copied instead, with exactly its production runtime
dependencies, as real files flat under `node_modules/<name>`.
- **Only this target's platform packages are copied.** Optional platform packages for other
systems, development dependencies, declarations, and source maps are left out. Two versions of
one package, a link inside a native package, or a required dependency for another platform fail
packaging.
- **`runtime` is read from the binaries themselves.** It records the Node-API version your addons
declare (or the exact Node.js ABI of an ABI-specific addon) and the newest glibc symbol version
any copied binary needs. A native target for a module with no native addon is refused: use
`any`.
- **A native package fits one environment.** A `linux-x64-glibc` artifact runs only on Linux,
x86-64, glibc hosts that meet its recorded glibc and Node-API minimums. The verifier checks all of
that before it loads anything. Other operating systems and CPUs, including ARM, are not supported
yet.
- **Your module's licence.** Your source `package.json` must contain a non-empty string SPDX
expression. The packager validates its syntax, rejects `UNLICENSED`, and copies it unchanged to
the required `ekklesia.package.json.license`. `--license-file` is also required: it names the
@@ -144,6 +164,18 @@ The archive holds one directory named with your module id:
supplies and says so). The module itself is not added to this notice file. Bundled Ekklesia
packages remain listed in `ekklesia.package.json` under `incorporates`; one applicable AGPL text
is sufficient for current official modules.
- **Reviewed licence supplements.** Some dependencies omit material that redistribution requires.
For one of those, the packager adds material from a reviewed supplement checked in with it, after
the package's own material and labelled as such
([ADR 0052](../architecture/decisions/0052-reviewed-license-supplements.md)).
- **Exact match.** A supplement is used only for an exact package name, version, and content
digest. The digest covers the redistributed files' relative paths and bytes, and is documented in
`packages/module-pack/src/supplements.ts`. Any mismatch fails.
- **What it does not do.** A supplement never changes the licence the package declares. The first
supplement covers `@img/sharp-libvips-linux-x64` 1.3.3, the libvips binary `sharp` loads. It
records upstream sources, but whether the LGPL and MPL source obligations for that binary are met
is an open determination. A package that carries it is not yet treated as publicly
distributable.
- **Deterministic.** The same packager version and the same built inputs produce the same archive,
byte for byte, wherever the checkout is.