feat: native linux-x64-glibc module packages and reviewed license supplements (ADR 0049, ADR 0052)
CI / verify (pull_request) Successful in 1m6s
CI / infrastructure-smoke (pull_request) Successful in 27s
CI / migration-smoke (pull_request) Successful in 1m54s
CI / image-build (pull_request) Successful in 29s
CI / deploy-smoke (same-origin) (pull_request) Successful in 43s
CI / deploy-smoke (split) (pull_request) Successful in 1m30s
CI / verify (pull_request) Successful in 1m6s
CI / infrastructure-smoke (pull_request) Successful in 27s
CI / migration-smoke (pull_request) Successful in 1m54s
CI / image-build (pull_request) Successful in 29s
CI / deploy-smoke (same-origin) (pull_request) Successful in 43s
CI / deploy-smoke (split) (pull_request) Successful in 1m30s
Implements ADR 0049's deferred native target and ADR 0052's reviewed license supplements. The proof case is a real module using sharp. packageFormat stays 1, and `any` is unchanged. Packager (@ekklesia/module-pack): - --target linux-x64-glibc. Native or platform-specific packages, found by the same generic rule that refuses them in `any`, stay outside the bundle. - Each native package's production closure is walked through its own dependencies, installed optionalDependencies that match the target, and required peers. It is copied as real files flat under node_modules/<name>, with no links, dev dependencies, declarations, source maps, or other platforms' packages. Missing or foreign-platform required dependencies, version conflicts, links, foreign machines, and Node.js range conflicts fail closed. - runtime.nodeApi, nodeAbi, and libcVersion are read from the binaries by a small ELF reader: the addon's node_api_module_get_api_version_v1 and the newest GLIBC_ symbol version needed. No shelling out. A native target with no addon is refused. - ADR 0052 supplements: package-provided material stays primary. An exact match on name, version, and ekklesia-content-v1 digest (sorted normalized paths with each file's size and SHA-256; no metadata) adds labelled material after the package's own. Any mismatch, a declared-licence conflict, or a malformed or unreviewed supplement file fails. Missing material with no supplement still fails. Supplement for @img/[email protected]: - 184 components and 143 unique texts, assembled from the exact upstream sources sharp-libvips v1.3.3 builds linux-x64 from: 27 C/C++ libraries, glib's gvdb, librsvg's Rust crates (reproduced with cargo tree), the Rust standard library at the rustc commit recorded in the binary, plus the LGPL-3.0, GPL-3.0, and MPL-2.0 texts. Each source is recorded with its SHA-256 in PROVENANCE.md. - It pins digest ekklesia-content-v1:sha256:50647b0b…, which is identical from the installed copy and from the registry tarball. - LGPL/MPL source-availability determinations stay open, as ADR 0052 states. Nothing claims they are met. Verifier (Core): - linux-x64-glibc is accepted only on a host whose platform, arch, glibc family and version, and Node-API version (from Node itself) match. - Every ELF file must be x86-64; every .node must be an ELF object. - `any` packages now also refuse node_modules/ and ELF files. - Contract: native metadata must state nodeApi or nodeAbi, and Linux targets libcVersion (ADR 0049). Proof: - A checked-in AGPL native fixture packs byte-identically from unrelated copies (8.86 MB) and verifies with Core's verifier. - Extracted elsewhere, it runs a sharp resize and PNG round trip in an empty-environment process. Every resolution stays in the package, and only the package's addon and libvips plus host system libraries load. Signage is untouched. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
f53506b3f0
commit
44d385cd7f
@@ -683,11 +683,25 @@ Module Package v1, a deterministic `.tar.gz` with its SHA-256.
|
||||
third parties, and writes the archive. The source module's string SPDX expression is required and
|
||||
becomes the package metadata's authoritative `license`; the packager neither infers nor downloads
|
||||
license material.
|
||||
- **Native target (`linux-x64-glibc`).** Native or platform-specific dependencies stay outside
|
||||
the bundle. Each one is copied with exactly its production runtime closure, as real files flat
|
||||
under the package's `node_modules/`, and only the platform packages that match the target are
|
||||
included. The package's `runtime` records the Node-API version and minimum glibc version, both
|
||||
read from the native binaries themselves.
|
||||
- **License supplements.** Where a redistributed dependency omits required licence material,
|
||||
reviewed supplements checked in with the packager add it
|
||||
([ADR 0052](decisions/0052-reviewed-license-supplements.md)). Each supplement matches one exact
|
||||
package name, version, and content digest; anything else fails closed.
|
||||
- **The verifier.** Core's `verifyModulePackage` reads the archive in memory and refuses links,
|
||||
special files, and escaping paths. It checks every digest and identity, the target and runtime,
|
||||
runs the platform's own module, schema, and migration validation, and loads the backend once in a
|
||||
short-lived process with an empty environment. Only then does it extract anything.
|
||||
- **What exists today.** Only the `any` target. A manifest-v8 ABI module carries no
|
||||
- **A native package's host checks.** It is accepted only on a host whose operating system, CPU,
|
||||
C library family and version, and Node-API version Node.js itself reports as compatible.
|
||||
- **ELF files.** Every ELF file in it must be built for the target's machine.
|
||||
- **What an `any` package may not hold:** `node_modules`, addons, and ELF files.
|
||||
- **What exists today.** The `any` and `linux-x64-glibc` targets. No other operating system or CPU
|
||||
target exists yet. A manifest-v8 ABI module carries no
|
||||
`transitional.databaseRuntime`; Drizzle may be bundled as its private implementation detail, and
|
||||
Core validates plain schema metadata without comparing ORM versions. A version 8 module that
|
||||
omits `database` remains on the temporary legacy facade and keeps ADR 0049's exact transitional
|
||||
|
||||
@@ -611,6 +611,31 @@ Ekklesia Module Package v1 (ADR 0049) is proven at three levels, all in `pnpm ve
|
||||
`..` paths, symbolic and hard links, device nodes and FIFOs, setuid and setgid entries, a second
|
||||
top-level directory, and a duplicate entry each fail with their own sanitized code. The backend
|
||||
load check runs in a real child process and selects the schema contract from the manifest.
|
||||
- `packages/module-pack/tests/native-target.test.ts`: the `linux-x64-glibc` target, with a real
|
||||
native dependency (`sharp`). The checked-in proof module (`tests/fixtures/native-module`) is
|
||||
packed, and the test proves:
|
||||
- the package carries exactly sharp's discovered runtime closure, as real files;
|
||||
- nothing from another platform, and no development dependency, is included;
|
||||
- `runtime` matches what the binaries declare;
|
||||
- the reviewed libvips supplement is merged and labelled;
|
||||
- two packs from unrelated copies are byte-identical.
|
||||
|
||||
Core's own verifier then accepts the package, and the extracted package runs a real image
|
||||
operation in a separate process with an empty environment. Node's resolution hooks and the
|
||||
process's memory map show that nothing resolves or loads from outside the package except the
|
||||
host's system libraries.
|
||||
- `packages/module-pack/tests/native-closure.test.ts` breaks the closure rules one at a time, with
|
||||
fake native packages built around the real sharp addon's bytes:
|
||||
- which platform packages are kept and which are left out;
|
||||
- missing and foreign-platform dependencies;
|
||||
- version conflicts, links, a foreign machine, and Node.js range conflicts;
|
||||
- an unneeded native target.
|
||||
- `packages/module-pack/tests/supplements.test.ts` proves ADR 0052's mechanism:
|
||||
- the documented content-digest algorithm;
|
||||
- that the loader refuses anything unreviewed;
|
||||
- that supplement material is merged after the package's own and labelled;
|
||||
- that a supplement for another version, digest, or licence fails closed;
|
||||
- that the shipped libvips supplement is intact and pins the installed package exactly.
|
||||
- `packages/module-pack/tests/`: the packager against generated fixture modules (payload contents,
|
||||
generated `package.json`, metadata, bundled backend with only built-in imports, refusal of native
|
||||
and platform-specific dependencies and of unknown or textless third-party licences, explicit
|
||||
|
||||
@@ -101,7 +101,7 @@ pnpm exec ekklesia-pack-module --module . --target any --out ../packages \
|
||||
--source-revision "$(git rev-parse HEAD)"
|
||||
```
|
||||
|
||||
It writes `<id>-<version>-any.tar.gz` and its `.sha256` line, and nothing else anywhere: it never
|
||||
It writes `<id>-<version>-<target>.tar.gz` and its `.sha256` line, and nothing else anywhere: it never
|
||||
installs, links, or deploys, so your workspace is left exactly as it was. Never run `pnpm deploy` in
|
||||
a workspace instead: it rewrites that workspace's install state to production-only.
|
||||
|
||||
@@ -118,7 +118,8 @@ The archive holds one directory named with your module id:
|
||||
├── dist/schema/index.js
|
||||
├── dist/chunks/ code your backend entrypoints share
|
||||
├── dist/ui/ … dist/display/ … dist/contributions/ your browser artifacts, exactly as built
|
||||
└── drizzle/ your migrations
|
||||
├── drizzle/ your migrations
|
||||
└── node_modules/ a native target only: native runtime dependencies, as real files
|
||||
```
|
||||
|
||||
- **Bundled backend.** Your server, schema, and jobs entrypoints are bundled with every
|
||||
@@ -129,9 +130,28 @@ The archive holds one directory named with your module id:
|
||||
your development `package.json`, any `workspace:` range, and links of any kind.
|
||||
- **Browser artifacts must be current.** The packager rebuilds them in memory and refuses an artifact
|
||||
on disk that differs, so run your build first.
|
||||
- **`any` is the only target today.** A dependency that is native or platform-specific (a compiled
|
||||
addon, a package that declares an operating system or CPU, or one that loads a per-platform binary
|
||||
such as `sharp`) cannot be in an `any` package, and packaging fails. Native targets come later.
|
||||
- **Targets.**
|
||||
- **`any`** is for pure-JavaScript modules. A dependency that is native or platform-specific (a
|
||||
compiled addon, a package that declares an operating system or CPU, or one that loads a
|
||||
per-platform binary such as `sharp`) cannot be in an `any` package, and packaging fails.
|
||||
- **`linux-x64-glibc`** is for modules with such a dependency, on Linux, x86-64, and glibc. Pack
|
||||
with `--target linux-x64-glibc` on a matching host, after installing your dependencies.
|
||||
- **What a `linux-x64-glibc` package contains.**
|
||||
- **Native packages stay outside the bundle.** Your ordinary JavaScript dependencies are bundled
|
||||
as before. Each native package is copied instead, with exactly its production runtime
|
||||
dependencies, as real files flat under `node_modules/<name>`.
|
||||
- **Only this target's platform packages are copied.** Optional platform packages for other
|
||||
systems, development dependencies, declarations, and source maps are left out. Two versions of
|
||||
one package, a link inside a native package, or a required dependency for another platform fail
|
||||
packaging.
|
||||
- **`runtime` is read from the binaries themselves.** It records the Node-API version your addons
|
||||
declare (or the exact Node.js ABI of an ABI-specific addon) and the newest glibc symbol version
|
||||
any copied binary needs. A native target for a module with no native addon is refused: use
|
||||
`any`.
|
||||
- **A native package fits one environment.** A `linux-x64-glibc` artifact runs only on Linux,
|
||||
x86-64, glibc hosts that meet its recorded glibc and Node-API minimums. The verifier checks all of
|
||||
that before it loads anything. Other operating systems and CPUs, including ARM, are not supported
|
||||
yet.
|
||||
- **Your module's licence.** Your source `package.json` must contain a non-empty string SPDX
|
||||
expression. The packager validates its syntax, rejects `UNLICENSED`, and copies it unchanged to
|
||||
the required `ekklesia.package.json.license`. `--license-file` is also required: it names the
|
||||
@@ -144,6 +164,18 @@ The archive holds one directory named with your module id:
|
||||
supplies and says so). The module itself is not added to this notice file. Bundled Ekklesia
|
||||
packages remain listed in `ekklesia.package.json` under `incorporates`; one applicable AGPL text
|
||||
is sufficient for current official modules.
|
||||
- **Reviewed licence supplements.** Some dependencies omit material that redistribution requires.
|
||||
For one of those, the packager adds material from a reviewed supplement checked in with it, after
|
||||
the package's own material and labelled as such
|
||||
([ADR 0052](../architecture/decisions/0052-reviewed-license-supplements.md)).
|
||||
- **Exact match.** A supplement is used only for an exact package name, version, and content
|
||||
digest. The digest covers the redistributed files' relative paths and bytes, and is documented in
|
||||
`packages/module-pack/src/supplements.ts`. Any mismatch fails.
|
||||
- **What it does not do.** A supplement never changes the licence the package declares. The first
|
||||
supplement covers `@img/sharp-libvips-linux-x64` 1.3.3, the libvips binary `sharp` loads. It
|
||||
records upstream sources, but whether the LGPL and MPL source obligations for that binary are met
|
||||
is an open determination. A package that carries it is not yet treated as publicly
|
||||
distributable.
|
||||
- **Deterministic.** The same packager version and the same built inputs produce the same archive,
|
||||
byte for byte, wherever the checkout is.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user