feat: corresponding-source artifacts for native packages that redistribute copyleft code (ADR 0055)

Ekklesia may publicly distribute a native Module Package containing LGPL or MPL components only beside a version-specific corresponding-source artifact. The artifact carries the exact source, patches, build definitions, and rebuild and relink instructions for those components. ADR 0055 (Accepted) records the policy: project compliance policy, not legal certification. It decides the item ADR 0052 deferred, and extends ADR 0049's distributable-release gate. Module Package v1, packageFormat 1, and both libvips supplements are unchanged. The mechanism is generic and lives in @ekklesia/module-pack: - Source recipes: reviewed data under corresponding-sources/<package>/<version>/. recipe.json names the exact package (name, version, content digest) and the exact supplement it was prepared with, and lists every component with its role, licence, licence material, and relationship to the binary. Every source file and patch is recorded by origin, size, and SHA-256; documents are hashed too, and every exclusion has a reason. Loading is strict and refuses anything unlisted. - Acquisition (the only network step) fetches each recorded file once into a cache keyed by SHA-256. It uses node:https with exact headers and no decoding, retries transient failures, and stores only bytes that match. - Construction is offline. From a built package it finds the covered native dependencies (by declared licence, or by a supplement component's licence), verifies every byte, and writes four sibling artifacts: the deterministic <package>-corresponding-source.tar.gz and its .sha256; <package>.redistribution.json, linking package, dependencies, supplement, recipe, and source by digest; and <package>.source-notice.txt, the release notice. - The public-distribution gate (verifyRedistribution, `ekklesia-corresponding-source verify`) approves only when the recipe, supplement, canonical archive, manifest, and metadata identify exactly these files. It refuses bundled copyleft code, and approves packages with no covered native code without an artifact. packModule is untouched. First complete proof: @img/[email protected] (sharp 0.35.5), from the reviewed supplement and provenance. - 383 components and 387 source files, about 208 MB: the sharp-libvips v1.3.4 build definitions, the 27 library archives, the five build-time patches, the 353 crates librsvg's resolved Cargo.lock records, and rust-src for nightly-2026-09-27 (rustc 75a75c3e0), which vendors std's dependencies. - BUILDING.md and two helpers point sharp-libvips's unchanged build logic at those files. - Followed in a Rocky Linux 8 container with networking disabled, they rebuilt libvips-cpp.so.8.18.7, byte-identically across two builds. Swapped into a verified, extracted native package, the rebuilt library was the one loaded, and sharp ran. - The native proof's source artifact, native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz, is 207242785 bytes, sha256 cc405492032b04a5391ab84e64f0e5e50f78e571ef7818204782c25a03eb50b7. It is byte-identical from unrelated directories and from a cache acquired fresh from upstream. Tests: - corresponding-source.test.ts runs in CI: synthetic inputs, every negative case, acquisition with retries and refusal, and determinism. - corresponding-source-native.test.ts checks the real recipe everywhere. With a verified cache (`pnpm test:corresponding-source`, which acquires once) it builds the real artifact twice, gates it, fails it closed, and runs sharp from the package. Docs: the module guide's distribution section, the README and architecture notes, and an ADR 0052 cross-reference. No Signage change, version bump, publication, or tag. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 23:17:35 -05:00
parent 4919db6168
commit e083c44e05
22 changed files with 14580 additions and 6 deletions
@@ -9,7 +9,9 @@
  further source: a reviewed supplement. Everything else in ADR 0049 is unchanged, as is
  `packageFormat`.
 - Related: [ADR 0048](0048-generic-image-and-installed-module-storage.md),
-  [ADR 0051](0051-module-package-license-material.md)
+  [ADR 0051](0051-module-package-license-material.md),
+  [ADR 0055](0055-corresponding-source-artifacts.md) (decides the source-obligation policy deferred
+  below)

 ## Context

@@ -0,0 +1,189 @@
+# 0055: Native redistributable packages carry corresponding-source artifacts
+
+- Status: Accepted
+- Date: 2026-10-09
+- Extends: [0049](0049-module-package-v1.md). ADR 0049 withholds distributable-release status from a
+  package until its licence handling is proven. This ADR adds what proving it means for a native
+  package that redistributes copyleft code: a sibling corresponding-source artifact, and a
+  fail-closed gate for public distribution. Module Package v1 is unchanged, and `packageFormat`
+  stays **1**.
+- Decides: the item [ADR 0052](0052-reviewed-license-supplements.md) deferred, "the legal and
+  compliance determination of source obligations for each redistributed binary, and how a
+  determination is recorded", as a project policy. Reviewed supplements are unchanged in purpose
+  and format.
+- Related: [ADR 0051](0051-module-package-license-material.md), whose rules for a module's own
+  licence are unchanged.
+
+## Context
+
+A native Module Package (ADR 0049) copies native dependencies into its `node_modules` exactly as
+built. The first one, the packager's native proof with sharp, redistributes
+`@img/sharp-libvips-linux-x64`. That is one shared object with about 29 libraries built into it.
+Several are under the GNU LGPL, and cairo and some Rust crates are under the MPL.
+
+ADR 0052 made the licence and notice side work. A reviewed supplement adds the licence, copyright,
+and notice files the package omits, with the exact upstream source each came from. But ADR 0052 was
+explicit that carrying material is not meeting obligations. The LGPL and the MPL also oblige whoever
+distributes the binary to make the corresponding source available. For a library linked into a
+larger work, that includes what a recipient needs to rebuild the library and use a modified version.
+Recording upstream URLs does not do that: URLs move, tags are rewritten, and a mirror is somebody
+else's promise.
+
+Until this was decided, every native package containing such components was blocked from public
+distribution, and the blocker had no defined way to be lifted.
+
+## Decision
+
+### Native packages may contain copyleft native dependencies
+
+A native Module Package may redistribute native dependencies under the LGPL, the MPL, or a similar
+licence with source obligations. It may be **distributed publicly** only when it is released
+together with a **corresponding-source artifact** that meets this ADR. A package whose artifact
+cannot be produced and verified is not approved for public distribution.
+
+This is Ekklesia's **project compliance policy**: a conservative rule the project holds itself to.
+It is not legal advice, and it is not legal certification. Repository review of the artifact, its
+inputs, and its instructions is project and code review. It does not replace professional legal
+review, and nothing built from this ADR may claim that it does.
+
+### Supplements and source artifacts do different jobs
+
+- **A reviewed supplement (ADR 0052)** identifies and verifies the licence and notice material for a
+  redistributed binary. It travels inside the package, in `THIRD_PARTY_NOTICES`.
+- **A corresponding-source artifact** carries the source, patches, build definitions, and rebuild
+  and relink instructions for the covered native components. It travels beside the package, as a
+  separate release artifact.
+
+A supplement alone never satisfies the source-availability policy. A source artifact never replaces
+the supplement or `THIRD_PARTY_NOTICES`.
+
+### What the artifact contains
+
+For each covered native dependency the package redistributes, the artifact contains:
+
+- the exact source the binary was built from, as bytes, not as references:
+  - its build definitions;
+  - the source archive of every library compiled into it;
+  - every patch the build applied;
+  - every source package a language package manager resolved for the build;
+  - the source of any runtime library the toolchain linked into it;
+- the licence and copyright material: the reviewed supplement, and the files inside the sources;
+- **instructions** that are technically actionable. They name what the artifact corresponds to and
+  its target, and say how to apply the patches and rebuild the covered library. They also say how a
+  rebuilt or modified library replaces the redistributed one in the installed module, and which
+  toolchain and configuration to use, or point to the included build definitions;
+- a deterministic, machine-readable manifest of every component and file, each with its digest,
+  plus the human-readable provenance.
+
+Components that are not part of the binary are left out, each with its reason recorded: build-only,
+test-only, or another platform's. The exception is a build-time input needed to rebuild the covered
+library, such as a source package a build script requires. That is included and marked as
+build-only. Host system libraries and general-purpose build tools are not redistributed, and the
+instructions name the versions they need.
+
+### It is tied to exact inputs, and verified before it is written
+
+- **Bound to the binary.** An artifact names each covered dependency by package name, version, and
+  the content digest a supplement is matched by (ADR 0052). It names the module identity and target
+  of the package it accompanies. An artifact for one version of a binary never satisfies another.
+- **From reviewed inputs only.** The inputs come from a checked-in, reviewed **source recipe** per
+  covered dependency version. The recipe records each component's name, version, origin,
+  relationship to the binary, and licence material, and each source file's and patch's size and
+  SHA-256.
+- **Acquisition is separate from construction.** Obtaining the recorded bytes may use the network,
+  in an explicit preparation step that stores them in a local cache keyed by content digest.
+  Constructing the artifact is offline. It reads only the cache and checked-in files, verifies every
+  byte against the recipe before writing anything, and never fetches or trusts a current upstream
+  response.
+- **Deterministic.** The same inputs produce the same artifact, byte for byte, in any working
+  directory: normalized relative paths, fixed ownership, modes, and timestamps, sorted entries, and
+  no machine-specific path.
+
+### Binary and source are linked by recorded digests
+
+A public release of a native package with covered components consists of sibling artifacts:
+
+- the Module Package archive and its SHA-256 (ADR 0049);
+- the corresponding-source archive and its SHA-256;
+- machine-readable **redistribution metadata** recording:
+  - the package archive's name and SHA-256;
+  - the module identity and target;
+  - each covered dependency's name, version, licence, and content digest;
+  - the supplement and recipe it was assembled from;
+  - the source archive's name and SHA-256, and the SHA-256 of its manifest;
+- a **release notice**, displayed alongside the binary wherever it is offered. It says, in
+  substance, that the release includes third-party software under licences including the GNU LGPL
+  or the MPL, and that corresponding source and build and relink materials for the covered native
+  dependencies are in the accompanying artifact, named by its exact file name and SHA-256.
+
+A reviewer can therefore tell exactly which source artifact corresponds to which redistributed
+binary, from digests, without guessing from file names. The source archive is never placed inside
+the Module Package. `THIRD_PARTY_NOTICES` stays as it is, and never duplicates the source.
+
+### Public distribution fails closed
+
+Building and testing packages stays unrestricted: the packager produces native packages exactly as
+before, for development, CI, and private use. **Public redistribution approval** is a separate gate.
+For a package intended for public distribution, the gate requires that:
+
+- the package's native dependency inventory is known;
+- every covered dependency has its reviewed supplement and source recipe;
+- the corresponding-source artifact is present;
+- every digest matches;
+- the artifact's manifest validates against the recipe and the archive's own contents;
+- the release notice names the exact artifact.
+
+If any requirement fails, approval fails. Bundled (not copied) code under such a licence is not
+handled by this mechanism, so the gate refuses it rather than approving it.
+
+### Retention, and no rewriting history
+
+- Ekklesia keeps a release's corresponding-source artifact for as long as it distributes, or
+  supports access to, the matching binary release. Nothing deletes one automatically while that
+  binary remains available.
+- The bytes of a source artifact for an existing binary release are never replaced under the same
+  identity.
+- A correction is a new release, or an explicitly versioned compliance artifact that the
+  redistribution metadata names. History is never silently replaced.
+
+### What this ADR does not do
+
+- It does not change any module's own licence, or ADR 0051's rules for it. The module's own source
+  obligations are met by the module's own repository, as before.
+- It does not make **runtime downloading** an acceptable workaround. Fetching a copyleft library at
+  run time instead of redistributing it moves the obligation rather than meeting it, makes
+  installations depend on a third party, and breaks ADR 0048's self-contained installed modules. It
+  is not the preferred path.
+- It does not change Module Package v1, the verifier's acceptance of a package, or the content of
+  existing supplements.
+
+## Consequences
+
+- Native modules that need copyleft native libraries, starting with sharp's libvips, can be
+  distributed publicly under a defined, checkable policy, instead of being blocked indefinitely.
+- Each covered dependency version needs a reviewed source recipe, as it needs a reviewed supplement.
+  A dependency bump needs both before public distribution works again.
+- Releases carry an artifact many times larger than the package itself (about 200 MB for libvips
+  1.3.4, against a 9 MB package), and Ekklesia commits to retaining it.
+- Constructing the artifact needs the verified source cache. Acquisition happens once per recipe,
+  not on every build.
+
+## Alternatives
+
+- **Treat supplements, source URLs, or a written offer as sufficient.** Rejected: URLs and offers
+  depend on third parties and on time. This policy requires the source itself, retained with the
+  release.
+- **Put the source inside the Module Package.** Rejected: installations would store hundreds of
+  megabytes they never need, and the package format would change for a distribution concern.
+- **Download sources while constructing the artifact.** Rejected: a release would depend on mutable
+  remote responses. Acquisition is explicit and verified, and construction is offline.
+- **Download native libraries at run time.** Rejected, as above.
+- **Per-dependency special cases in code.** Rejected for the same reason as in ADR 0052: recipes are
+  reviewed data, and the mechanism knows no particular library.
+
+## Deferred
+
+- Hosting and serving source artifacts for published module releases, and the release CI that runs
+  the gate and keeps the verified source cache.
+- Corresponding source for bundled copyleft code, if a module ever needs it.
+- Recipes provided by module repositories, alongside the question ADR 0052 deferred for supplements.
@@ -63,3 +63,4 @@ ADRs capture durable choices that affect more than one package or phase.
 - [0052: Reviewed license supplements cover redistributed dependencies whose packages omit required material](0052-reviewed-license-supplements.md) — Proposed
 - [0053: A module builds outside the monorepo, from Ekklesia packages published over the npm registry protocol](0053-standalone-module-development-packages.md) — Accepted
 - [0054: Ekklesia's module-development packages are publicly readable before launch](0054-public-package-reads-before-launch.md) — Accepted
+- [0055: Native redistributable packages carry corresponding-source artifacts](0055-corresponding-source-artifacts.md) — Accepted
@@ -692,6 +692,12 @@ Module Package v1, a deterministic `.tar.gz` with its SHA-256.
    reviewed supplements checked in with the packager add it
    ([ADR 0052](decisions/0052-reviewed-license-supplements.md)). Each supplement matches one exact
    package name, version, and content digest; anything else fails closed.
+  - **Corresponding source.** A native package that redistributes code under the LGPL, the MPL, or
+    a similar licence is approved for public distribution only together with a sibling
+    corresponding-source artifact. The artifact is built offline from a reviewed source recipe and a
+    verified source cache, and linked to the package by digest
+    ([ADR 0055](decisions/0055-corresponding-source-artifacts.md)). Building and testing packages is
+    unaffected.
 - **The verifier.** Core's `verifyModulePackage` reads the archive in memory and refuses links,
  special files, and escaping paths. It checks every digest and identity, the target and runtime,
  runs the platform's own module, schema, and migration validation, and loads the backend once in a
@@ -296,9 +296,10 @@ The archive holds one directory named with your module id:
  - **What it does not do.** A supplement never changes the licence the package declares. The
    supplements cover `@img/sharp-libvips-linux-x64` 1.3.3 and 1.3.4, the libvips binaries that
    `sharp` 0.35.4 and 0.35.5 load; each matches only its own version and digest, and the packager's
-    native proof builds with sharp 0.35.5. They record upstream sources, but whether the LGPL and MPL
-    source obligations for those binaries are met is an open determination. A package that carries
-    one is not yet treated as publicly distributable.
+    native proof builds with sharp 0.35.5. A supplement identifies and verifies licence and notice
+    material only. Source availability for copyleft native code is a separate requirement, met by a
+    corresponding-source artifact
+    ([Distributing a native package publicly](#distributing-a-native-package-publicly)).
 - **Deterministic.** The same packager version and the same built inputs produce the same archive,
  byte for byte, wherever the checkout is.

@@ -324,6 +325,103 @@ installed package normally has no source, so the platform cannot check it the sa
 module's own CI is where its boundary rules, its production dependencies, and its behaviour inside
 Ekklesia are proven ([ADR 0047](../architecture/decisions/0047-module-ci-harness.md)).

+#### Distributing a native package publicly
+
+Building and testing a native package needs nothing beyond the packager. **Distributing it
+publicly** does, when it redistributes native code under the GNU LGPL, the MPL, or a similar licence
+with source obligations
+([ADR 0055](../architecture/decisions/0055-corresponding-source-artifacts.md)).
+
+- **Covered dependencies.** A native dependency is covered when its declared licence, or any
+  component its reviewed supplement records inside it, carries source obligations. libvips is one:
+  it declares `LGPL-3.0-or-later`, and cairo and several Rust crates inside it are under the MPL.
+- **The policy.** A package with covered dependencies may be distributed publicly only together with
+  a corresponding-source artifact. The artifact holds the exact source, patches, build definitions,
+  and rebuild and relink instructions for those dependencies, as bytes rather than links. If the
+  artifact cannot be produced and verified, the package is not approved for public distribution.
+  This is Ekklesia's project compliance policy, not legal advice or certification. A package that
+  passes the gate meets Ekklesia's redistribution policy, and nothing more is claimed.
+- **Source recipes.** For each covered dependency version, a reviewed recipe is checked in beside
+  the supplements, under `packages/module-pack/corresponding-sources/<package>/<version>/`. It holds
+  `recipe.json` and its documents, notably `BUILDING.md`. The recipe names the exact package (name,
+  version, content digest) and the exact supplement it was prepared with. It lists every component
+  with its role (`compiled`, `build-input`, or `build-definition`), licence, licence material, and
+  relationship to the binary. Every source file and patch is recorded by origin, size, and SHA-256,
+  and every exclusion has a reason. Like supplements, recipes are reviewed data, and the mechanism
+  knows no particular library.
+
+Releasing a native package publicly takes three steps after packaging:
+
+```bash
+# 1. Acquisition: the only step that uses the network. It fetches each recorded file once into a
+#    cache keyed by SHA-256, refusing any response that differs from the recipe.
+pnpm exec ekklesia-corresponding-source acquire --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
+  --cache ~/.cache/ekklesia/corresponding-sources
+
+# 2. Construction: offline. Every byte is verified against the recipe before anything is written.
+pnpm exec ekklesia-corresponding-source build --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
+  --cache ~/.cache/ekklesia/corresponding-sources --out ../packages
+
+# 3. The public-distribution gate. Exit status 0 means approved.
+pnpm exec ekklesia-corresponding-source verify --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
+  --source ../packages/<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz \
+  --metadata ../packages/<id>-<version>-linux-x64-glibc.redistribution.json
+```
+
+Construction writes four sibling release artifacts beside the package. The source archive never goes
+inside the Module Package, and `THIRD_PARTY_NOTICES` is unchanged.
+
+```text
+<id>-<version>-linux-x64-glibc.tar.gz                           the Module Package (packager)
+<id>-<version>-linux-x64-glibc.tar.gz.sha256
+<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz      the corresponding source
+<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz.sha256
+<id>-<version>-linux-x64-glibc.redistribution.json              binary ↔ source, by digest
+<id>-<version>-linux-x64-glibc.source-notice.txt                the release notice
+```
+
+- **The source archive** is deterministic: sorted entries, fixed modes, ownership, and timestamps,
+  and no machine-specific path. The same package and recipe produce the same bytes in any directory.
+  - `corresponding-source.json` records every file with its SHA-256, and `SHA256SUMS` repeats them.
+  - Each covered dependency has its own `dependencies/<package>/<version>/` directory, with
+    `BUILDING.md`, the recipe, `PROVENANCE.md`, the supplement, and `sources/` and `patches/`.
+- **The redistribution metadata** answers "which exact source corresponds to this exact binary?"
+  from digests. It records:
+  - the package file and its SHA-256, the module identity, and the target;
+  - each covered dependency's version and content digest, and the supplement and recipe it was
+    assembled from;
+  - the source archive's file name and SHA-256, and the SHA-256 of its manifest.
+- **The release notice** must be shown alongside the binary wherever it is offered. It states that
+  the release includes third-party software under licences including the GNU LGPL and the MPL, and
+  that corresponding source and build and relink materials for the covered native dependencies are
+  in the accompanying artifact. It names that artifact by its exact file name and SHA-256.
+- **Fail-closed.** The gate refuses a package when:
+  - a covered dependency has no recipe, or a recipe or supplement does not match its exact bytes;
+  - the source archive is missing, non-canonical, or differs from its manifest;
+  - the archive holds a file the manifest does not record, or a generated file names a local path;
+  - the metadata or the notice does not name these exact files.
+
+  It also refuses bundled (not copied) copyleft code, which this mechanism does not cover. A package
+  with no covered native code is approved without an artifact.
+- **Retention.** Keep a release's source artifact for as long as its binary release is distributed
+  or supported. Never delete one automatically while the binary remains available. Never replace its
+  bytes under the same name. A correction is a new release, or an explicitly versioned compliance
+  artifact.
+
+**Available today:** a recipe for `@img/sharp-libvips-linux-x64` 1.3.4, used by sharp 0.35.5. Its
+artifact is about 200 MB: 27 library archives, 5 patches, 353 Rust crates, and the Rust standard
+library source of the recorded toolchain. Following its `BUILDING.md` rebuilds
+`libvips-cpp.so.8.18.7` from that source alone. libvips 1.3.3 (sharp 0.35.4) has a supplement but no
+recipe, so a package with it builds and tests as before but is not approved for public distribution.
+`pnpm test:corresponding-source` runs the whole proof for the native proof module: acquisition into a
+verified cache, two byte-identical builds, the gate, and sharp running from the package.
+
+**Adding a native dependency** follows the same path as libvips:
+- curate and review its supplement (ADR 0052);
+- curate and review its source recipe and `BUILDING.md`, recording every source input the
+  dependency's build fetches;
+- prove both with the native packaging tests and `pnpm test:corresponding-source`.
+
 ### License

 Ekklesia is licensed under the GNU Affero General Public License, version 3 or (at your option) any