feat: corresponding-source artifacts for native packages that redistribute copyleft code (ADR 0055) #86

Merged

Zayin_Admin merged 2 commits from feat/corresponding-source into main 2026-10-09 04:34:50 +00:00

Conversation 1 Commits 2 Files Changed 22

+14582 -8

ekklesia-bot commented 2026-10-09 04:19:40 +00:00

Member

Copy Link

Copy Source

Implements the LGPL/MPL redistribution decision. Ekklesia may publicly distribute a native Module Package containing copyleft native components only beside a version-specific corresponding-source artifact with the exact source, patches, build definitions, and rebuild/relink instructions. Without that artifact, the package isn't approved. This is a conservative project compliance policy, not legal advice or certification.

ADR 0055 (Accepted): Native redistributable packages carry corresponding-source artifacts

• Extends ADR 0049's distributable-release gate. Module Package v1 and packageFormat 1 are unchanged; the source archive is a sibling artifact, never inside the package.
• Decides the item ADR 0052 deferred (the source-obligation policy). Supplements keep their purpose of identifying and verifying licence material; alone, they don't satisfy source availability.
• ADR 0051's rules for a module's own licence are unchanged.
• It records:
  • exact-input binding;
  • reviewed recipes, with acquisition kept separate from offline, deterministic construction;
  • binary/source linkage by recorded digests;
  • the release notice;
  • retention, with no rewriting of history;
  • a fail-closed public-distribution gate;
  • runtime downloading is not a workaround.
• Marked Accepted because the implementation proves the decision end to end, including an actual rebuild from the artifact (below).

The mechanism (generic, in @ekklesia/module-pack)

• Source recipes live in corresponding-sources/<package>/<version>/: recipe.json plus hashed documents. A recipe binds:

  • the exact package (name, version, content digest);
  • the exact supplement.json (sha256).

  Each component records its role (compiled / build-input / build-definition), licence, supplement licence material, and relationship to the binary. Every source file and patch is recorded by origin, size and sha256, and every exclusion has a reason. Loading is strict and refuses unlisted or changed files. Nothing in code knows about libvips.

• Acquisition (ekklesia-corresponding-source acquire) is the only network step. It fetches into a cache keyed by sha256 (<cache>/sha256/<hex>) and stores only exact bytes, with bounded retries for transient upstream failures. It uses node:https with explicit headers and no content decoding, because the platform fetch was refused (HTTP 406) by freedesktop.org's GitLab.

• Construction (build) is offline. From a built package it finds the covered native dependencies (by declared licence, or by any supplement component's licence), verifies every byte, and writes:

  • <package>-corresponding-source.tar.gz (module-pack's deterministic ustar+gzip) and its .sha256;
  • <package>.redistribution.json, with package sha256, module and target, each covered dependency's version, content digest, supplement and recipe sha256, the source file name and sha256, and the manifest sha256;
  • <package>.source-notice.txt, the release notice naming the exact file and SHA-256.

  The archive holds a README, corresponding-source.json (every component and every file, by sha256), SHA256SUMS, and dependencies/<pkg>/<version>/ with BUILDING.md, recipe.json, PROVENANCE.md, licenses/ (the supplement), sources/, patches/ and build helpers.

• The gate (verify / verifyRedistribution) approves only when the recipe, supplement, canonical archive, manifest and metadata identify these exact files. It refuses bundled copyleft code, which this mechanism doesn't cover, and approves packages with no covered native code without an artifact. packModule (build/test packaging) is untouched.

First complete proof: @img/[email protected] (sharp 0.35.5)

• Recipe: 383 components and 387 files, 207.7 MB of source, all verified:
  • sharp-libvips v1.3.4 build definitions;
  • the 27 library archives;
  • the 5 build-time patches (glib, mozjpeg, librsvg, libultrahdr, libvips);
  • all 353 crates in librsvg's resolved Cargo.lock: 147 compiled, 17 build-only, and 189 Cargo needs to resolve the workspace offline;
  • rust-src for nightly-2026-09-27 (= rustc 75a75c3e0, the commit recorded in the binary), which vendors std's dependencies.
• gvdb maps to glib's archive. Exclusions: proxy-libintl, unlinked std crates, host system libraries, and the toolchain. Every supplement component maps to a source component.
• BUILDING.md covers how the binary was built, how to rebuild it (prepare-offline.sh points sharp-libvips's unchanged build/posix.sh at the artifact's files and the vendored crates, and pins the nightly toolchain), and how to swap in a rebuilt library (the addon's RPATH resolves node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.8.18.7).
• Actually rebuilt from the artifact. In the sharp-libvips Rocky Linux 8 container, with networking disabled (docker run --network none), posix.sh rebuilt libvips-cpp.so.8.18.7. It was byte-identical across two builds (sha256 2217773a…), with the same SONAME, NEEDED set and GLIBC_2.28 floor, and an identical versions.json.
  • The first attempt failed because Cargo needs the whole workspace lockfile. That's why the recipe carries all 353 crates rather than only the 164-crate build closure.
• Relink proven. Swapping the rebuilt library into a Core-verified, extracted native package: the process mapped the replaced file, and sharp's resize and PNG round trip succeeded.
• The native proof's artifact: native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz, 207,242,785 bytes, sha256 cc405492032b04a5391ab84e64f0e5e50f78e571ef7818204782c25a03eb50b7, manifest sha256 d5af6a8f…, for package sha256 061bc1ae….
  • It is byte-identical from unrelated directories, and from a cache acquired fresh from upstream (387 files fetched through the tool, all matching).

Tests

• corresponding-source.test.ts (CI, synthetic inputs): a deterministic build, and approval exactly as built. It also fails closed on each of these:

  • a missing archive or metadata;
  • a wrong source digest, or a corrupted archive;
  • a notice that doesn't name the artifact;
  • a wrong native package digest;
  • a wrong component version, or a missing component;
  • changed patch bytes, even with the manifest relisted;
  • a manifest naming absent bytes;
  • unrecorded material in the archive;
  • a local path in a generated file;
  • non-canonical entry order;
  • an artifact for another dependency version (the 1.3.3-for-1.3.4 case);
  • bundled copyleft code;
  • a missing or mismatched recipe.

  It also covers strict recipe loading, licence classification, and acquisition (fetch-once, retry, refusal of changed upstream bytes, nothing stored). I checked that the tests catch the canonical-order check, the local-path check and the manifest comparison when each is removed.

• corresponding-source-native.test.ts: the real recipe checks run everywhere. With a verified cache (pnpm test:corresponding-source, which acquires once) it:

  • packs the sharp 0.35.5 proof;
  • builds the artifact twice (byte-identical);
  • has the gate approve it, then refuse it when it's removed or corrupted;
  • verifies the package with Core, and runs sharp from it.

• Results:

  • module-pack: 111 tests pass (3 cache-gated tests skip without a cache);
  • pnpm test:corresponding-source: 4 of 4;
  • pnpm verify: passes;
  • strict scripts/test-package-release.sh: passes. module-pack now ships corresponding-sources/, the ekklesia-corresponding-source bin and its dist; byte-identical rebuild; standalone consumer proof.

Preserved

• The libvips 1.3.3 and 1.3.4 supplements are byte-for-byte unchanged (the recipe refers to 1.3.4's by sha256).
• libvips 1.3.3 has no recipe, so a package with it builds and tests as before but isn't approved for public distribution.
• No Signage change, version bump, publication, or tag.

Remaining before Signage PR #12

1. A release containing this module-pack (0.2.2 or 0.3.0, per the release guide) and its publication.
2. Signage PR #12:
   • move to that module-pack, with sharp 0.35.5 and the matching release-age exemptions;
   • add the license field and LICENSE file;
   • pack linux-x64-glibc, then acquire, build and verify the corresponding source in its release CI;
   • re-enable the integration job.
3. Hosting and retention for published module releases' source artifacts (ADR 0055 Deferred), and a verified source cache for release CI.

Also: ADR 0052 accepted

Every decision in ADR 0052 is implemented and tested: exact name, version and digest matching that fails closed; the documented, host-independent digest; package-first, labelled, deterministic merging; no licence override, with a consistency check; no network; packageFormat 1. The source-obligation determination it deferred is decided here by ADR 0055. Commit 98d93cc changes only its status line and index entry; the decision text is untouched.

🤖 Generated with Claude Code

Implements the LGPL/MPL redistribution decision. Ekklesia **may** publicly distribute a native Module Package containing copyleft native components **only beside** a version-specific corresponding-source artifact with the exact source, patches, build definitions, and rebuild/relink instructions. Without that artifact, the package isn't approved. This is a conservative project compliance policy, not legal advice or certification. ## ADR 0055 (Accepted): Native redistributable packages carry corresponding-source artifacts - **Extends ADR 0049's distributable-release gate.** Module Package v1 and `packageFormat` 1 are unchanged; the source archive is a **sibling** artifact, never inside the package. - **Decides the item ADR 0052 deferred** (the source-obligation policy). Supplements keep their purpose of identifying and verifying licence material; alone, they don't satisfy source availability. - **ADR 0051's rules for a module's own licence are unchanged.** - It records: - exact-input binding; - reviewed recipes, with acquisition kept separate from offline, deterministic construction; - binary/source linkage by recorded digests; - the release notice; - retention, with no rewriting of history; - a fail-closed public-distribution gate; - runtime downloading is not a workaround. - Marked Accepted because the implementation proves the decision end to end, including an actual rebuild from the artifact (below). ## The mechanism (generic, in `@ekklesia/module-pack`) - **Source recipes** live in `corresponding-sources/<package>/<version>/`: `recipe.json` plus hashed documents. A recipe binds: - the exact package (name, version, content digest); - the exact `supplement.json` (sha256). Each component records its role (`compiled` / `build-input` / `build-definition`), licence, supplement licence material, and relationship to the binary. Every source file and patch is recorded by origin, size and sha256, and every exclusion has a reason. Loading is strict and refuses unlisted or changed files. Nothing in code knows about libvips. - **Acquisition** (`ekklesia-corresponding-source acquire`) is the only network step. It fetches into a cache keyed by sha256 (`<cache>/sha256/<hex>`) and stores only exact bytes, with bounded retries for transient upstream failures. It uses `node:https` with explicit headers and no content decoding, because the platform `fetch` was refused (HTTP 406) by freedesktop.org's GitLab. - **Construction** (`build`) is offline. From a built package it finds the covered native dependencies (by declared licence, or by any supplement component's licence), verifies every byte, and writes: - `<package>-corresponding-source.tar.gz` (module-pack's deterministic ustar+gzip) and its `.sha256`; - `<package>.redistribution.json`, with package sha256, module and target, each covered dependency's version, content digest, supplement and recipe sha256, the source file name and sha256, and the manifest sha256; - `<package>.source-notice.txt`, the release notice naming the exact file and SHA-256. The archive holds a README, `corresponding-source.json` (every component and every file, by sha256), `SHA256SUMS`, and `dependencies/<pkg>/<version>/` with `BUILDING.md`, `recipe.json`, `PROVENANCE.md`, `licenses/` (the supplement), `sources/`, `patches/` and build helpers. - **The gate** (`verify` / `verifyRedistribution`) approves only when the recipe, supplement, canonical archive, manifest and metadata identify **these exact files**. It refuses bundled copyleft code, which this mechanism doesn't cover, and approves packages with no covered native code without an artifact. `packModule` (build/test packaging) is untouched. ## First complete proof: `@img/[email protected]` (sharp 0.35.5) - **Recipe:** 383 components and 387 files, 207.7 MB of source, all verified: - sharp-libvips v1.3.4 build definitions; - the 27 library archives; - the 5 build-time patches (glib, mozjpeg, librsvg, libultrahdr, libvips); - all 353 crates in librsvg's resolved `Cargo.lock`: 147 compiled, 17 build-only, and 189 Cargo needs to resolve the workspace offline; - `rust-src` for `nightly-2026-09-27` (= rustc `75a75c3e0`, the commit recorded in the binary), which vendors std's dependencies. - **gvdb** maps to glib's archive. **Exclusions:** proxy-libintl, unlinked std crates, host system libraries, and the toolchain. Every supplement component maps to a source component. - **`BUILDING.md`** covers how the binary was built, how to rebuild it (`prepare-offline.sh` points sharp-libvips's unchanged `build/posix.sh` at the artifact's files and the vendored crates, and pins the nightly toolchain), and how to swap in a rebuilt library (the addon's RPATH resolves `node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.8.18.7`). - **Actually rebuilt from the artifact.** In the sharp-libvips Rocky Linux 8 container, **with networking disabled** (`docker run --network none`), `posix.sh` rebuilt `libvips-cpp.so.8.18.7`. It was byte-identical across two builds (sha256 `2217773a…`), with the same SONAME, NEEDED set and GLIBC_2.28 floor, and an identical `versions.json`. - The first attempt failed because Cargo needs the whole workspace lockfile. That's why the recipe carries all 353 crates rather than only the 164-crate build closure. - **Relink proven.** Swapping the rebuilt library into a Core-verified, extracted native package: the process mapped the replaced file, and sharp's resize and PNG round trip succeeded. - **The native proof's artifact:** `native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz`, 207,242,785 bytes, **sha256 `cc405492032b04a5391ab84e64f0e5e50f78e571ef7818204782c25a03eb50b7`**, manifest sha256 `d5af6a8f…`, for package sha256 `061bc1ae…`. - It is byte-identical from unrelated directories, and from a cache acquired fresh from upstream (387 files fetched through the tool, all matching). ## Tests - **`corresponding-source.test.ts`** (CI, synthetic inputs): a deterministic build, and approval exactly as built. It also fails closed on each of these: - a missing archive or metadata; - a wrong source digest, or a corrupted archive; - a notice that doesn't name the artifact; - a wrong native package digest; - a wrong component version, or a missing component; - changed patch bytes, even with the manifest relisted; - a manifest naming absent bytes; - unrecorded material in the archive; - a local path in a generated file; - non-canonical entry order; - an artifact for another dependency version (the 1.3.3-for-1.3.4 case); - bundled copyleft code; - a missing or mismatched recipe. It also covers strict recipe loading, licence classification, and acquisition (fetch-once, retry, refusal of changed upstream bytes, nothing stored). I checked that the tests catch the canonical-order check, the local-path check and the manifest comparison when each is removed. - **`corresponding-source-native.test.ts`:** the real recipe checks run everywhere. With a verified cache (`pnpm test:corresponding-source`, which acquires once) it: - packs the sharp 0.35.5 proof; - builds the artifact twice (byte-identical); - has the gate approve it, then refuse it when it's removed or corrupted; - verifies the package with Core, and runs sharp from it. - **Results:** - module-pack: 111 tests pass (3 cache-gated tests skip without a cache); - `pnpm test:corresponding-source`: 4 of 4; - `pnpm verify`: passes; - strict `scripts/test-package-release.sh`: passes. module-pack now ships `corresponding-sources/`, the `ekklesia-corresponding-source` bin and its dist; byte-identical rebuild; standalone consumer proof. ## Preserved - The libvips 1.3.3 and 1.3.4 supplements are byte-for-byte unchanged (the recipe refers to 1.3.4's by sha256). - libvips 1.3.3 has no recipe, so a package with it builds and tests as before but isn't approved for public distribution. - No Signage change, version bump, publication, or tag. ## Remaining before Signage PR #12 1. A release containing this module-pack (0.2.2 or 0.3.0, per the release guide) and its publication. 2. Signage PR #12: - move to that module-pack, with sharp 0.35.5 and the matching release-age exemptions; - add the `license` field and `LICENSE` file; - pack linux-x64-glibc, then acquire, build and verify the corresponding source in its release CI; - re-enable the integration job. 3. Hosting and retention for published module releases' source artifacts (ADR 0055 *Deferred*), and a verified source cache for release CI. ## Also: ADR 0052 accepted Every decision in ADR 0052 is implemented and tested: exact name, version and digest matching that fails closed; the documented, host-independent digest; package-first, labelled, deterministic merging; no licence override, with a consistency check; no network; `packageFormat` 1. The source-obligation determination it deferred is decided here by ADR 0055. Commit `98d93cc` changes only its status line and index entry; the decision text is untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

ekklesia-bot added 1 commit 2026-10-09 04:19:40 +00:00

feat: corresponding-source artifacts for native packages that redistribute copyleft code (ADR 0055) ... e083c44e05

Ekklesia may publicly distribute a native Module Package containing LGPL or MPL components only beside
a version-specific corresponding-source artifact. The artifact carries the exact source, patches,
build definitions, and rebuild and relink instructions for those components. ADR 0055 (Accepted)
records the policy: project compliance policy, not legal certification. It decides the item ADR 0052
deferred, and extends ADR 0049's distributable-release gate. Module Package v1, packageFormat 1, and
both libvips supplements are unchanged.

The mechanism is generic and lives in @ekklesia/module-pack:
- Source recipes: reviewed data under corresponding-sources/<package>/<version>/. recipe.json names
  the exact package (name, version, content digest) and the exact supplement it was prepared with,
  and lists every component with its role, licence, licence material, and relationship to the
  binary. Every source file and patch is recorded by origin, size, and SHA-256; documents are hashed
  too, and every exclusion has a reason. Loading is strict and refuses anything unlisted.
- Acquisition (the only network step) fetches each recorded file once into a cache keyed by
  SHA-256. It uses node:https with exact headers and no decoding, retries transient failures, and
  stores only bytes that match.
- Construction is offline. From a built package it finds the covered native dependencies (by
  declared licence, or by a supplement component's licence), verifies every byte, and writes four
  sibling artifacts: the deterministic <package>-corresponding-source.tar.gz and its .sha256;
  <package>.redistribution.json, linking package, dependencies, supplement, recipe, and source by
  digest; and <package>.source-notice.txt, the release notice.
- The public-distribution gate (verifyRedistribution, `ekklesia-corresponding-source verify`)
  approves only when the recipe, supplement, canonical archive, manifest, and metadata identify
  exactly these files. It refuses bundled copyleft code, and approves packages with no covered
  native code without an artifact. packModule is untouched.

First complete proof: @img/[email protected] (sharp 0.35.5), from the reviewed supplement
and provenance.
- 383 components and 387 source files, about 208 MB: the sharp-libvips v1.3.4 build definitions,
  the 27 library archives, the five build-time patches, the 353 crates librsvg's resolved Cargo.lock
  records, and rust-src for nightly-2026-09-27 (rustc 75a75c3e0), which vendors std's dependencies.
- BUILDING.md and two helpers point sharp-libvips's unchanged build logic at those files.
- Followed in a Rocky Linux 8 container with networking disabled, they rebuilt
  libvips-cpp.so.8.18.7, byte-identically across two builds. Swapped into a verified, extracted
  native package, the rebuilt library was the one loaded, and sharp ran.
- The native proof's source artifact, native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz,
  is 207242785 bytes, sha256 cc405492032b04a5391ab84e64f0e5e50f78e571ef7818204782c25a03eb50b7. It is
  byte-identical from unrelated directories and from a cache acquired fresh from upstream.

Tests:
- corresponding-source.test.ts runs in CI: synthetic inputs, every negative case, acquisition with
  retries and refusal, and determinism.
- corresponding-source-native.test.ts checks the real recipe everywhere. With a verified cache
  (`pnpm test:corresponding-source`, which acquires once) it builds the real artifact twice, gates
  it, fails it closed, and runs sharp from the package.

Docs: the module guide's distribution section, the README and architecture notes, and an ADR 0052
cross-reference.

No Signage change, version bump, publication, or tag.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Zayin_Admin added 1 commit 2026-10-09 04:24:19 +00:00

docs: accept ADR 0052, whose supplement mechanism is fully implemented ... 98d93cc12d

Every decision in ADR 0052 is implemented and tested in @ekklesia/module-pack:
- package material first, and a supplement only adds to it;
- reviewed, generic supplement data, checked in with the packager;
- an exact name, version, and content-digest match, failing closed on any mismatch;
- the documented, host-independent digest;
- a deterministic, labelled merge;
- no licence override, with a consistency check;
- no network during packaging;
- packageFormat 1.

The libvips 1.3.3 and 1.3.4 supplements are curated and shipped. The source-obligation determination
it deferred is now decided by ADR 0055. Only the status line and the index entry change; the decision
text is untouched.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

Zayin_Admin approved these changes 2026-10-09 04:34:37 +00:00

Zayin_Admin merged commit 9ad2d5b564 into main 2026-10-09 04:34:50 +00:00

Zayin_Admin deleted branch feat/corresponding-source 2026-10-09 04:34:50 +00:00

ekklesia-bot referenced this pull request 2026-10-09 21:58:35 +00:00

chore(release): prepare 0.2.2 #87

Sign in to join this conversation.

Reviewers

No Reviewers

Zayin_Admin

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

Joey Zayin_Admin ekklesia-bot

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Zayin/Ekklesia#86