feat: corresponding-source artifacts for native packages that redistribute copyleft code (ADR 0055)
CI / verify (pull_request) Successful in 1m3s
CI / infrastructure-smoke (pull_request) Successful in 36s
CI / migration-smoke (pull_request) Successful in 1m53s
CI / package-release (pull_request) Successful in 36s
CI / package-publication (pull_request) Canceled after 0s
CI / deploy-smoke (same-origin) (pull_request) Canceled after 0s
CI / deploy-smoke (split) (pull_request) Canceled after 0s
CI / image-build (pull_request) Canceled after 30s

Ekklesia may publicly distribute a native Module Package containing LGPL or MPL components only beside
a version-specific corresponding-source artifact. The artifact carries the exact source, patches,
build definitions, and rebuild and relink instructions for those components. ADR 0055 (Accepted)
records the policy: project compliance policy, not legal certification. It decides the item ADR 0052
deferred, and extends ADR 0049's distributable-release gate. Module Package v1, packageFormat 1, and
both libvips supplements are unchanged.

The mechanism is generic and lives in @ekklesia/module-pack:
- Source recipes: reviewed data under corresponding-sources/<package>/<version>/. recipe.json names
  the exact package (name, version, content digest) and the exact supplement it was prepared with,
  and lists every component with its role, licence, licence material, and relationship to the
  binary. Every source file and patch is recorded by origin, size, and SHA-256; documents are hashed
  too, and every exclusion has a reason. Loading is strict and refuses anything unlisted.
- Acquisition (the only network step) fetches each recorded file once into a cache keyed by
  SHA-256. It uses node:https with exact headers and no decoding, retries transient failures, and
  stores only bytes that match.
- Construction is offline. From a built package it finds the covered native dependencies (by
  declared licence, or by a supplement component's licence), verifies every byte, and writes four
  sibling artifacts: the deterministic <package>-corresponding-source.tar.gz and its .sha256;
  <package>.redistribution.json, linking package, dependencies, supplement, recipe, and source by
  digest; and <package>.source-notice.txt, the release notice.
- The public-distribution gate (verifyRedistribution, `ekklesia-corresponding-source verify`)
  approves only when the recipe, supplement, canonical archive, manifest, and metadata identify
  exactly these files. It refuses bundled copyleft code, and approves packages with no covered
  native code without an artifact. packModule is untouched.

First complete proof: @img/[email protected] (sharp 0.35.5), from the reviewed supplement
and provenance.
- 383 components and 387 source files, about 208 MB: the sharp-libvips v1.3.4 build definitions,
  the 27 library archives, the five build-time patches, the 353 crates librsvg's resolved Cargo.lock
  records, and rust-src for nightly-2026-09-27 (rustc 75a75c3e0), which vendors std's dependencies.
- BUILDING.md and two helpers point sharp-libvips's unchanged build logic at those files.
- Followed in a Rocky Linux 8 container with networking disabled, they rebuilt
  libvips-cpp.so.8.18.7, byte-identically across two builds. Swapped into a verified, extracted
  native package, the rebuilt library was the one loaded, and sharp ran.
- The native proof's source artifact, native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz,
  is 207242785 bytes, sha256 cc405492032b04a5391ab84e64f0e5e50f78e571ef7818204782c25a03eb50b7. It is
  byte-identical from unrelated directories and from a cache acquired fresh from upstream.

Tests:
- corresponding-source.test.ts runs in CI: synthetic inputs, every negative case, acquisition with
  retries and refusal, and determinism.
- corresponding-source-native.test.ts checks the real recipe everywhere. With a verified cache
  (`pnpm test:corresponding-source`, which acquires once) it builds the real artifact twice, gates
  it, fails it closed, and runs sharp from the package.

Docs: the module guide's distribution section, the README and architecture notes, and an ADR 0052
cross-reference.

No Signage change, version bump, publication, or tag.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-10-08 23:17:35 -05:00
co-authored by Claude Opus 5.5
parent 4919db6168
commit e083c44e05
22 changed files with 14580 additions and 6 deletions
+4 -1
View File
@@ -59,7 +59,10 @@ supplied `LICENSE` bytes; dependency material remains separate in `THIRD_PARTY_N
packages may bundle a module-owned Drizzle version without coupling it to Core;
legacy-v8 packages retain their transitional exact-version check. Where a redistributed dependency
omits licence material, the packager may add reviewed licence supplements
([ADR 0052](docs/architecture/decisions/0052-reviewed-license-supplements.md)). Purge, other
([ADR 0052](docs/architecture/decisions/0052-reviewed-license-supplements.md)). A native package that
redistributes LGPL or MPL code is distributed publicly only with its corresponding-source artifact,
which `ekklesia-corresponding-source` builds and gates
([ADR 0055](docs/architecture/decisions/0055-corresponding-source-artifacts.md)). Purge, other
operating-system or CPU targets, signing, and any installer or marketplace do not exist yet.
A module can be built in its own repository from Ekklesia's public module-development packages
@@ -9,7 +9,9 @@
further source: a reviewed supplement. Everything else in ADR 0049 is unchanged, as is
`packageFormat`.
- Related: [ADR 0048](0048-generic-image-and-installed-module-storage.md),
[ADR 0051](0051-module-package-license-material.md)
[ADR 0051](0051-module-package-license-material.md),
[ADR 0055](0055-corresponding-source-artifacts.md) (decides the source-obligation policy deferred
below)
## Context
@@ -0,0 +1,189 @@
# 0055: Native redistributable packages carry corresponding-source artifacts
- Status: Accepted
- Date: 2026-10-09
- Extends: [0049](0049-module-package-v1.md). ADR 0049 withholds distributable-release status from a
package until its licence handling is proven. This ADR adds what proving it means for a native
package that redistributes copyleft code: a sibling corresponding-source artifact, and a
fail-closed gate for public distribution. Module Package v1 is unchanged, and `packageFormat`
stays **1**.
- Decides: the item [ADR 0052](0052-reviewed-license-supplements.md) deferred, "the legal and
compliance determination of source obligations for each redistributed binary, and how a
determination is recorded", as a project policy. Reviewed supplements are unchanged in purpose
and format.
- Related: [ADR 0051](0051-module-package-license-material.md), whose rules for a module's own
licence are unchanged.
## Context
A native Module Package (ADR 0049) copies native dependencies into its `node_modules` exactly as
built. The first one, the packager's native proof with sharp, redistributes
`@img/sharp-libvips-linux-x64`. That is one shared object with about 29 libraries built into it.
Several are under the GNU LGPL, and cairo and some Rust crates are under the MPL.
ADR 0052 made the licence and notice side work. A reviewed supplement adds the licence, copyright,
and notice files the package omits, with the exact upstream source each came from. But ADR 0052 was
explicit that carrying material is not meeting obligations. The LGPL and the MPL also oblige whoever
distributes the binary to make the corresponding source available. For a library linked into a
larger work, that includes what a recipient needs to rebuild the library and use a modified version.
Recording upstream URLs does not do that: URLs move, tags are rewritten, and a mirror is somebody
else's promise.
Until this was decided, every native package containing such components was blocked from public
distribution, and the blocker had no defined way to be lifted.
## Decision
### Native packages may contain copyleft native dependencies
A native Module Package may redistribute native dependencies under the LGPL, the MPL, or a similar
licence with source obligations. It may be **distributed publicly** only when it is released
together with a **corresponding-source artifact** that meets this ADR. A package whose artifact
cannot be produced and verified is not approved for public distribution.
This is Ekklesia's **project compliance policy**: a conservative rule the project holds itself to.
It is not legal advice, and it is not legal certification. Repository review of the artifact, its
inputs, and its instructions is project and code review. It does not replace professional legal
review, and nothing built from this ADR may claim that it does.
### Supplements and source artifacts do different jobs
- **A reviewed supplement (ADR 0052)** identifies and verifies the licence and notice material for a
redistributed binary. It travels inside the package, in `THIRD_PARTY_NOTICES`.
- **A corresponding-source artifact** carries the source, patches, build definitions, and rebuild
and relink instructions for the covered native components. It travels beside the package, as a
separate release artifact.
A supplement alone never satisfies the source-availability policy. A source artifact never replaces
the supplement or `THIRD_PARTY_NOTICES`.
### What the artifact contains
For each covered native dependency the package redistributes, the artifact contains:
- the exact source the binary was built from, as bytes, not as references:
- its build definitions;
- the source archive of every library compiled into it;
- every patch the build applied;
- every source package a language package manager resolved for the build;
- the source of any runtime library the toolchain linked into it;
- the licence and copyright material: the reviewed supplement, and the files inside the sources;
- **instructions** that are technically actionable. They name what the artifact corresponds to and
its target, and say how to apply the patches and rebuild the covered library. They also say how a
rebuilt or modified library replaces the redistributed one in the installed module, and which
toolchain and configuration to use, or point to the included build definitions;
- a deterministic, machine-readable manifest of every component and file, each with its digest,
plus the human-readable provenance.
Components that are not part of the binary are left out, each with its reason recorded: build-only,
test-only, or another platform's. The exception is a build-time input needed to rebuild the covered
library, such as a source package a build script requires. That is included and marked as
build-only. Host system libraries and general-purpose build tools are not redistributed, and the
instructions name the versions they need.
### It is tied to exact inputs, and verified before it is written
- **Bound to the binary.** An artifact names each covered dependency by package name, version, and
the content digest a supplement is matched by (ADR 0052). It names the module identity and target
of the package it accompanies. An artifact for one version of a binary never satisfies another.
- **From reviewed inputs only.** The inputs come from a checked-in, reviewed **source recipe** per
covered dependency version. The recipe records each component's name, version, origin,
relationship to the binary, and licence material, and each source file's and patch's size and
SHA-256.
- **Acquisition is separate from construction.** Obtaining the recorded bytes may use the network,
in an explicit preparation step that stores them in a local cache keyed by content digest.
Constructing the artifact is offline. It reads only the cache and checked-in files, verifies every
byte against the recipe before writing anything, and never fetches or trusts a current upstream
response.
- **Deterministic.** The same inputs produce the same artifact, byte for byte, in any working
directory: normalized relative paths, fixed ownership, modes, and timestamps, sorted entries, and
no machine-specific path.
### Binary and source are linked by recorded digests
A public release of a native package with covered components consists of sibling artifacts:
- the Module Package archive and its SHA-256 (ADR 0049);
- the corresponding-source archive and its SHA-256;
- machine-readable **redistribution metadata** recording:
- the package archive's name and SHA-256;
- the module identity and target;
- each covered dependency's name, version, licence, and content digest;
- the supplement and recipe it was assembled from;
- the source archive's name and SHA-256, and the SHA-256 of its manifest;
- a **release notice**, displayed alongside the binary wherever it is offered. It says, in
substance, that the release includes third-party software under licences including the GNU LGPL
or the MPL, and that corresponding source and build and relink materials for the covered native
dependencies are in the accompanying artifact, named by its exact file name and SHA-256.
A reviewer can therefore tell exactly which source artifact corresponds to which redistributed
binary, from digests, without guessing from file names. The source archive is never placed inside
the Module Package. `THIRD_PARTY_NOTICES` stays as it is, and never duplicates the source.
### Public distribution fails closed
Building and testing packages stays unrestricted: the packager produces native packages exactly as
before, for development, CI, and private use. **Public redistribution approval** is a separate gate.
For a package intended for public distribution, the gate requires that:
- the package's native dependency inventory is known;
- every covered dependency has its reviewed supplement and source recipe;
- the corresponding-source artifact is present;
- every digest matches;
- the artifact's manifest validates against the recipe and the archive's own contents;
- the release notice names the exact artifact.
If any requirement fails, approval fails. Bundled (not copied) code under such a licence is not
handled by this mechanism, so the gate refuses it rather than approving it.
### Retention, and no rewriting history
- Ekklesia keeps a release's corresponding-source artifact for as long as it distributes, or
supports access to, the matching binary release. Nothing deletes one automatically while that
binary remains available.
- The bytes of a source artifact for an existing binary release are never replaced under the same
identity.
- A correction is a new release, or an explicitly versioned compliance artifact that the
redistribution metadata names. History is never silently replaced.
### What this ADR does not do
- It does not change any module's own licence, or ADR 0051's rules for it. The module's own source
obligations are met by the module's own repository, as before.
- It does not make **runtime downloading** an acceptable workaround. Fetching a copyleft library at
run time instead of redistributing it moves the obligation rather than meeting it, makes
installations depend on a third party, and breaks ADR 0048's self-contained installed modules. It
is not the preferred path.
- It does not change Module Package v1, the verifier's acceptance of a package, or the content of
existing supplements.
## Consequences
- Native modules that need copyleft native libraries, starting with sharp's libvips, can be
distributed publicly under a defined, checkable policy, instead of being blocked indefinitely.
- Each covered dependency version needs a reviewed source recipe, as it needs a reviewed supplement.
A dependency bump needs both before public distribution works again.
- Releases carry an artifact many times larger than the package itself (about 200 MB for libvips
1.3.4, against a 9 MB package), and Ekklesia commits to retaining it.
- Constructing the artifact needs the verified source cache. Acquisition happens once per recipe,
not on every build.
## Alternatives
- **Treat supplements, source URLs, or a written offer as sufficient.** Rejected: URLs and offers
depend on third parties and on time. This policy requires the source itself, retained with the
release.
- **Put the source inside the Module Package.** Rejected: installations would store hundreds of
megabytes they never need, and the package format would change for a distribution concern.
- **Download sources while constructing the artifact.** Rejected: a release would depend on mutable
remote responses. Acquisition is explicit and verified, and construction is offline.
- **Download native libraries at run time.** Rejected, as above.
- **Per-dependency special cases in code.** Rejected for the same reason as in ADR 0052: recipes are
reviewed data, and the mechanism knows no particular library.
## Deferred
- Hosting and serving source artifacts for published module releases, and the release CI that runs
the gate and keeps the verified source cache.
- Corresponding source for bundled copyleft code, if a module ever needs it.
- Recipes provided by module repositories, alongside the question ADR 0052 deferred for supplements.
+1
View File
@@ -63,3 +63,4 @@ ADRs capture durable choices that affect more than one package or phase.
- [0052: Reviewed license supplements cover redistributed dependencies whose packages omit required material](0052-reviewed-license-supplements.md) — Proposed
- [0053: A module builds outside the monorepo, from Ekklesia packages published over the npm registry protocol](0053-standalone-module-development-packages.md) — Accepted
- [0054: Ekklesia's module-development packages are publicly readable before launch](0054-public-package-reads-before-launch.md) — Accepted
- [0055: Native redistributable packages carry corresponding-source artifacts](0055-corresponding-source-artifacts.md) — Accepted
@@ -692,6 +692,12 @@ Module Package v1, a deterministic `.tar.gz` with its SHA-256.
reviewed supplements checked in with the packager add it
([ADR 0052](decisions/0052-reviewed-license-supplements.md)). Each supplement matches one exact
package name, version, and content digest; anything else fails closed.
- **Corresponding source.** A native package that redistributes code under the LGPL, the MPL, or
a similar licence is approved for public distribution only together with a sibling
corresponding-source artifact. The artifact is built offline from a reviewed source recipe and a
verified source cache, and linked to the package by digest
([ADR 0055](decisions/0055-corresponding-source-artifacts.md)). Building and testing packages is
unaffected.
- **The verifier.** Core's `verifyModulePackage` reads the archive in memory and refuses links,
special files, and escaping paths. It checks every digest and identity, the target and runtime,
runs the platform's own module, schema, and migration validation, and loads the backend once in a
+101 -3
View File
@@ -296,9 +296,10 @@ The archive holds one directory named with your module id:
- **What it does not do.** A supplement never changes the licence the package declares. The
supplements cover `@img/sharp-libvips-linux-x64` 1.3.3 and 1.3.4, the libvips binaries that
`sharp` 0.35.4 and 0.35.5 load; each matches only its own version and digest, and the packager's
native proof builds with sharp 0.35.5. They record upstream sources, but whether the LGPL and MPL
source obligations for those binaries are met is an open determination. A package that carries
one is not yet treated as publicly distributable.
native proof builds with sharp 0.35.5. A supplement identifies and verifies licence and notice
material only. Source availability for copyleft native code is a separate requirement, met by a
corresponding-source artifact
([Distributing a native package publicly](#distributing-a-native-package-publicly)).
- **Deterministic.** The same packager version and the same built inputs produce the same archive,
byte for byte, wherever the checkout is.
@@ -324,6 +325,103 @@ installed package normally has no source, so the platform cannot check it the sa
module's own CI is where its boundary rules, its production dependencies, and its behaviour inside
Ekklesia are proven ([ADR 0047](../architecture/decisions/0047-module-ci-harness.md)).
#### Distributing a native package publicly
Building and testing a native package needs nothing beyond the packager. **Distributing it
publicly** does, when it redistributes native code under the GNU LGPL, the MPL, or a similar licence
with source obligations
([ADR 0055](../architecture/decisions/0055-corresponding-source-artifacts.md)).
- **Covered dependencies.** A native dependency is covered when its declared licence, or any
component its reviewed supplement records inside it, carries source obligations. libvips is one:
it declares `LGPL-3.0-or-later`, and cairo and several Rust crates inside it are under the MPL.
- **The policy.** A package with covered dependencies may be distributed publicly only together with
a corresponding-source artifact. The artifact holds the exact source, patches, build definitions,
and rebuild and relink instructions for those dependencies, as bytes rather than links. If the
artifact cannot be produced and verified, the package is not approved for public distribution.
This is Ekklesia's project compliance policy, not legal advice or certification. A package that
passes the gate meets Ekklesia's redistribution policy, and nothing more is claimed.
- **Source recipes.** For each covered dependency version, a reviewed recipe is checked in beside
the supplements, under `packages/module-pack/corresponding-sources/<package>/<version>/`. It holds
`recipe.json` and its documents, notably `BUILDING.md`. The recipe names the exact package (name,
version, content digest) and the exact supplement it was prepared with. It lists every component
with its role (`compiled`, `build-input`, or `build-definition`), licence, licence material, and
relationship to the binary. Every source file and patch is recorded by origin, size, and SHA-256,
and every exclusion has a reason. Like supplements, recipes are reviewed data, and the mechanism
knows no particular library.
Releasing a native package publicly takes three steps after packaging:
```bash
# 1. Acquisition: the only step that uses the network. It fetches each recorded file once into a
# cache keyed by SHA-256, refusing any response that differs from the recipe.
pnpm exec ekklesia-corresponding-source acquire --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
--cache ~/.cache/ekklesia/corresponding-sources
# 2. Construction: offline. Every byte is verified against the recipe before anything is written.
pnpm exec ekklesia-corresponding-source build --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
--cache ~/.cache/ekklesia/corresponding-sources --out ../packages
# 3. The public-distribution gate. Exit status 0 means approved.
pnpm exec ekklesia-corresponding-source verify --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
--source ../packages/<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz \
--metadata ../packages/<id>-<version>-linux-x64-glibc.redistribution.json
```
Construction writes four sibling release artifacts beside the package. The source archive never goes
inside the Module Package, and `THIRD_PARTY_NOTICES` is unchanged.
```text
<id>-<version>-linux-x64-glibc.tar.gz the Module Package (packager)
<id>-<version>-linux-x64-glibc.tar.gz.sha256
<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz the corresponding source
<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz.sha256
<id>-<version>-linux-x64-glibc.redistribution.json binary ↔ source, by digest
<id>-<version>-linux-x64-glibc.source-notice.txt the release notice
```
- **The source archive** is deterministic: sorted entries, fixed modes, ownership, and timestamps,
and no machine-specific path. The same package and recipe produce the same bytes in any directory.
- `corresponding-source.json` records every file with its SHA-256, and `SHA256SUMS` repeats them.
- Each covered dependency has its own `dependencies/<package>/<version>/` directory, with
`BUILDING.md`, the recipe, `PROVENANCE.md`, the supplement, and `sources/` and `patches/`.
- **The redistribution metadata** answers "which exact source corresponds to this exact binary?"
from digests. It records:
- the package file and its SHA-256, the module identity, and the target;
- each covered dependency's version and content digest, and the supplement and recipe it was
assembled from;
- the source archive's file name and SHA-256, and the SHA-256 of its manifest.
- **The release notice** must be shown alongside the binary wherever it is offered. It states that
the release includes third-party software under licences including the GNU LGPL and the MPL, and
that corresponding source and build and relink materials for the covered native dependencies are
in the accompanying artifact. It names that artifact by its exact file name and SHA-256.
- **Fail-closed.** The gate refuses a package when:
- a covered dependency has no recipe, or a recipe or supplement does not match its exact bytes;
- the source archive is missing, non-canonical, or differs from its manifest;
- the archive holds a file the manifest does not record, or a generated file names a local path;
- the metadata or the notice does not name these exact files.
It also refuses bundled (not copied) copyleft code, which this mechanism does not cover. A package
with no covered native code is approved without an artifact.
- **Retention.** Keep a release's source artifact for as long as its binary release is distributed
or supported. Never delete one automatically while the binary remains available. Never replace its
bytes under the same name. A correction is a new release, or an explicitly versioned compliance
artifact.
**Available today:** a recipe for `@img/sharp-libvips-linux-x64` 1.3.4, used by sharp 0.35.5. Its
artifact is about 200 MB: 27 library archives, 5 patches, 353 Rust crates, and the Rust standard
library source of the recorded toolchain. Following its `BUILDING.md` rebuilds
`libvips-cpp.so.8.18.7` from that source alone. libvips 1.3.3 (sharp 0.35.4) has a supplement but no
recipe, so a package with it builds and tests as before but is not approved for public distribution.
`pnpm test:corresponding-source` runs the whole proof for the native proof module: acquisition into a
verified cache, two byte-identical builds, the gate, and sharp running from the package.
**Adding a native dependency** follows the same path as libvips:
- curate and review its supplement (ADR 0052);
- curate and review its source recipe and `BUILDING.md`, recording every source input the
dependency's build fetches;
- prove both with the native packaging tests and `pnpm test:corresponding-source`.
### License
Ekklesia is licensed under the GNU Affero General Public License, version 3 or (at your option) any
+1
View File
@@ -43,6 +43,7 @@
"test:migrations:integration": "pnpm build && bash scripts/test-migrations-integration.sh",
"test:root": "vitest run --config vitest.config.ts",
"test:package-release": "bash scripts/test-package-release.sh",
"test:corresponding-source": "bash scripts/test-corresponding-source.sh",
"test:package-publication": "bash scripts/test-package-publication.sh",
"test:watch": "vitest",
"typecheck": "turbo run typecheck typecheck:root",
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env node
// Committed launcher, deliberately not generated: see ekklesia-pack-module.mjs for why. It defers to
// the compiled CLI at run time.
import { fileURLToPath } from "node:url";
const compiledCli = new URL("../dist/corresponding-source-cli.js", import.meta.url);
try {
await import(compiledCli.href);
} catch (error) {
if (error?.code === "ERR_MODULE_NOT_FOUND") {
process.stderr.write(
`@ekklesia/module-pack is not built yet (${fileURLToPath(compiledCli)}).\n` +
"Run `pnpm build` from the repository root first.\n",
);
process.exitCode = 1;
} else {
throw error;
}
}
@@ -0,0 +1,144 @@
# Rebuilding `@img/sharp-libvips-linux-x64` 1.3.4
This directory is the corresponding source for one redistributed native binary. These instructions
say how to rebuild it from the source here, and how to use a rebuilt or modified library in place of
the redistributed one. They are technical instructions. They are not a legal opinion, and they have
not been legally certified.
## What this corresponds to
| | |
| --- | --- |
| Package | `@img/sharp-libvips-linux-x64` 1.3.4, from the npm registry |
| Declared licence | `LGPL-3.0-or-later` |
| Content digest | `ekklesia-content-v1:sha256:2927fe11493461fa34847191d0b9b1c6441007c3e99f306e5708cb07fb0a5e59` |
| Binary | `lib/libvips-cpp.so.8.18.7` (SONAME `libvips-cpp.so.8.18.7`) |
| Target | Linux, x86-64, glibc 2.28 or newer |
| Used by | `sharp` 0.35.5, through `@img/sharp-linux-x64` 0.35.5 |
`lib/libvips-cpp.so.8.18.7` is one shared object. libvips and the libraries it uses are built as
static libraries and linked into it: 27 C and C++ libraries, glib's bundled gvdb, librsvg with its
Rust crates, and the Rust standard library. `recipe.json` lists every component, its licence, its
source file and patches with their SHA-256, and its relationship to the binary. `PROVENANCE.md`
records how all of this was established, and `licenses/` holds the reviewed licence and notice
material.
## What is here
- `sources/sharp-libvips-1.3.4.tar.gz`: the build definitions that produced the binary. They are
`build.sh`, `build/posix.sh`, `platforms/linux-x64/` (a Dockerfile and its toolchain files), and
`versions.properties`.
- `sources/*.tar.*`: the source archive of every compiled library, exactly as `build/posix.sh`
downloads it.
- `patches/`: the five patches `build/posix.sh` applies: glib, mozjpeg, librsvg, libultrahdr, and
libvips.
- `sources/crates/`: all 353 Rust crates librsvg's resolved `Cargo.lock` records.
- 147 are compiled into the binary.
- 17 are needed only to build it: build-script dependencies, and one procedural macro that adds no
code.
- 189 serve librsvg's other workspace members, its tests, or other platforms. Cargo resolves the
whole workspace lockfile even when it builds one package, so it needs them present to build
offline. None of them is compiled into the binary.
`recipe.json` records each crate's role.
- `sources/rust/rust-src-nightly-2026-09-27.tar.xz`: the standard-library source of the Rust toolchain
that built librsvg, `rustc 1.101.0-nightly (75a75c3e0 2026-09-26)`.
- `build/`: the resolved librsvg `Cargo.lock`, and the two scripts below that point the unchanged build
logic at these files.
Every file's SHA-256 is in `recipe.json` and in the archive's top-level `corresponding-source.json`.
Check them before building.
## How the original was built
sharp-libvips builds the binary in a Rocky Linux 8 container, which pins glibc 2.28:
```sh
./build.sh linux-x64
# = docker build --pull -t vips-dev-linux-x64 platforms/linux-x64
# docker run --rm -v "$PWD:/packaging" vips-dev-linux-x64 sh -c /packaging/build/posix.sh
```
`platforms/linux-x64/Dockerfile` installs the toolchain: gcc-toolset-15, cmake, nasm, pkgconf, meson
and ninja (from pip), a nightly Rust toolchain through rustup, and cargo-c. `build/posix.sh` then
downloads each library's archive, applies the patches, builds every library as a static library into
`/target`, and links libvips-cpp. It packs the result as `sharp-libvips-linux-x64.tar.gz`, whose
`lib/libvips-cpp.so.8.18.7` is the file the npm package ships. Meson runs with
`wrap_mode = 'nofallback'` (`platforms/linux-x64/meson.ini`), so it downloads no subproject. The only
forced fallback, gvdb, builds from the copy inside glib's archive.
## Rebuilding from this source
You need a Linux x86-64 host with Docker. Building the container image needs network access to fetch
the toolchain (Rocky Linux packages, rustup, pip, and crates.io for cargo-c). Source for the library
itself comes only from this directory.
1. **Unpack the build definitions.**
```sh
tar -xzf sources/sharp-libvips-1.3.4.tar.gz # creates sharp-libvips-1.3.4/
```
2. **Point the build at this source.**
```sh
sh build/prepare-offline.sh sharp-libvips-1.3.4
```
It copies the archives and patches into `sharp-libvips-1.3.4/offline/`. It unpacks the crates into
a vendored directory source; Cargo checks each against the lockfile's checksum. It then makes three
mechanical edits:
- in `build/posix.sh`, `CURL` becomes `offline/offline-curl.sh`. That script answers each URL the
script would download with its local copy, using `build/url-map.tsv`, and fails for any other URL;
- in `build/posix.sh`, `cargo update --workspace` becomes a copy of the resolved `Cargo.lock`. Cargo
is set offline and uses the vendored crates;
- in `platforms/linux-x64/Dockerfile`, the Rust toolchain is pinned to `nightly-2026-09-27`, the
toolchain whose `rustc` built the original. The upstream file installs whatever nightly is
current.
Nothing that is compiled changes: every patch is still applied by `build/posix.sh`, exactly as
upstream.
3. **Build.**
```sh
cd sharp-libvips-1.3.4
./build.sh linux-x64
```
The result is `sharp-libvips-1.3.4/sharp-libvips-linux-x64.tar.gz`. Its
`lib/libvips-cpp.so.8.18.7` is the rebuilt library. A full build takes from tens of minutes to an
hour, depending on the host.
**Modifying a component.** Before step 3, change the source the build uses. Then `build/posix.sh`
compiles your version in place of the original:
- for a C or C++ library, replace its archive in `offline/files/` under the same name, or add a patch
to the matching step of `build/posix.sh`;
- for a Rust crate, edit it under `offline/vendor/`. The `.cargo-checksum.json` files list no
per-file checksums, so an edited crate still builds.
The result is not byte-identical to the redistributed binary: a compiler toolchain, timestamps, and
paths that were not recorded can all differ. It is built from the same source, by the same
definitions, for the same target.
## Using a rebuilt or modified library
sharp's native addon, `@img/sharp-linux-x64/lib/sharp-linux-x64-0.35.5.node`, links
`libvips-cpp.so.8.18.7` dynamically. Its RPATH begins with `$ORIGIN/../../sharp-libvips-linux-x64/lib`.
In an installed or extracted Ekklesia Module Package, that resolves to:
```text
<module directory>/node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.8.18.7
```
To use your library, replace that file with your build's `lib/libvips-cpp.so.8.18.7`. Keep the file
name and SONAME, and keep libvips's C++ API compatible with libvips 8.18, which sharp 0.35.5 is
compiled against. Then restart the process that loads the module. No other file changes, and the
addon does not need rebuilding.
`ldd <module directory>/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64-0.35.5.node` shows which
`libvips-cpp.so.8.18.7` the addon resolves.
A module package's own `ekklesia.package.json` records the SHA-256 of each file it shipped, so a
modified tree is no longer the package Ekklesia verified at installation. Install it as your own
build.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,25 @@
#!/bin/sh
# Stands in for curl in sharp-libvips's build/posix.sh, so the build reads only this artifact's
# sources. Given the URL posix.sh would download, it prints the recorded local copy, or with -O
# writes it to the current directory under the URL's file name. A URL with no recorded copy fails:
# the build never reaches the network for source.
set -eu
write=""
url=""
for argument in "$@"; do
case "$argument" in
-O) write=yes ;;
-*) ;;
*) url="$argument" ;;
esac
done
file="$(awk -F '\t' -v url="$url" '$1 == url { print $2 }' /packaging/offline/url-map.tsv)"
if [ -z "$file" ]; then
echo "offline-curl: no recorded source for $url" >&2
exit 22
fi
if [ -n "$write" ]; then
cp "/packaging/offline/files/$file" "./${url##*/}"
else
cat "/packaging/offline/files/$file"
fi
@@ -0,0 +1,61 @@
#!/bin/sh
# Prepares an extracted sharp-libvips v1.3.4 source tree to build lib/libvips-cpp.so.8.18.7 from this
# artifact's sources alone. BUILDING.md describes each step.
#
# sh build/prepare-offline.sh <extracted sharp-libvips-1.3.4 directory>
#
# Only the source acquisition changes; nothing that is compiled does:
# - every archive and patch build/posix.sh downloads is read from this artifact (offline-curl.sh);
# - librsvg's Rust crates come from this artifact as a vendored directory source, and its lockfile
# is the resolved one recorded here instead of a fresh `cargo update` against crates.io;
# - platforms/linux-x64/Dockerfile installs the exact nightly Rust toolchain the binary records.
set -eu
here="$(cd "$(dirname "$0")/.." && pwd)"
tree="$(cd "$1" && pwd)"
[ -f "$tree/build/posix.sh" ] && [ -f "$tree/platforms/linux-x64/Dockerfile" ] || {
echo "prepare-offline: $1 is not an extracted sharp-libvips source tree." >&2
exit 1
}
offline="$tree/offline"
rm -rf "$offline"
mkdir -p "$offline/files" "$offline/vendor"
cp "$here"/sources/*.tar.* "$here"/patches/*.patch "$offline/files/"
cp "$tree/THIRD-PARTY-NOTICES.md" "$offline/files/THIRD-PARTY-NOTICES.md"
cp "$here/build/url-map.tsv" "$here/build/Cargo.lock" "$offline/"
cp "$here/build/offline-curl.sh" "$offline/offline-curl.sh"
chmod 755 "$offline/offline-curl.sh"
for crate in "$here"/sources/crates/*.crate; do
tar -xzf "$crate" -C "$offline/vendor"
name="$(basename "$crate" .crate)"
checksum="$(sha256sum "$crate" | cut -d ' ' -f 1)"
printf '{"files":{},"package":"%s"}\n' "$checksum" > "$offline/vendor/$name/.cargo-checksum.json"
done
cat > "$offline/cargo-config.toml" <<'CONFIG'
[source.crates-io]
replace-with = "vendored-sources"
[source.vendored-sources]
directory = "/packaging/offline/vendor"
[net]
offline = true
CONFIG
posix="$tree/build/posix.sh"
grep -q '^CURL="curl ' "$posix" && grep -q '^cargo update --workspace$' "$posix" || {
echo "prepare-offline: build/posix.sh is not the v1.3.4 script this artifact was prepared for." >&2
exit 1
}
sed -i \
-e 's|^CURL="curl .*"$|CURL="/packaging/offline/offline-curl.sh"\nexport CARGO_NET_OFFLINE=true\nmkdir -p "${CARGO_HOME:-$HOME/.cargo}" \&\& cp /packaging/offline/cargo-config.toml "${CARGO_HOME:-$HOME/.cargo}/config.toml"|' \
-e 's|^cargo update --workspace$|cp /packaging/offline/Cargo.lock Cargo.lock|' \
"$posix"
dockerfile="$tree/platforms/linux-x64/Dockerfile"
grep -q -- '--default-toolchain nightly ' "$dockerfile" || {
echo "prepare-offline: platforms/linux-x64/Dockerfile is not the v1.3.4 one." >&2
exit 1
}
sed -i 's|--default-toolchain nightly |--default-toolchain nightly-2026-09-27 |' "$dockerfile"
echo "prepare-offline: $tree builds from this artifact's sources."
@@ -0,0 +1,33 @@
https://cairographics.org/releases/cairo-1.18.6.tar.xz cairo-1.18.6.tar.xz
https://cairographics.org/releases/pixman-0.46.4.tar.gz pixman-0.46.4.tar.gz
https://download.gnome.org/sources/glib/2.90/glib-2.90.0.tar.xz glib-2.90.0.tar.xz
https://download.gnome.org/sources/librsvg/2.63/librsvg-2.63.2.tar.xz librsvg-2.63.2.tar.xz
https://download.gnome.org/sources/libxml2/2.15/libxml2-2.15.4.tar.xz libxml2-2.15.4.tar.xz
https://download.gnome.org/sources/pango/1.58/pango-1.58.2.tar.xz pango-1.58.2.tar.xz
https://gist.github.com/kleisauke/284d685efa00908da99ea6afbaaf39ae/raw/bdad5489a61c217850631571caf57f5db6ea8b2c/glib-without-gregex.patch glib-without-gregex.patch
https://gist.githubusercontent.com/lovell/313a6901e9db1bf285f2a1f1180499e4/raw/3988223c7dfa4d22745d9392034b0117abef1446/libvips-cpp-soversion.patch libvips-cpp-soversion.patch
https://github.com/GNOME/librsvg/commit/9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch librsvg-9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch
https://github.com/dloebl/cgif/archive/v0.5.4.tar.gz cgif-0.5.4.tar.gz
https://github.com/freetype/freetype/archive/VER-2-14-3.tar.gz freetype-2.14.3.tar.gz
https://github.com/fribidi/fribidi/releases/download/v1.0.17/fribidi-1.0.17.tar.xz fribidi-1.0.17.tar.xz
https://github.com/google/highway/archive/1.4.0.tar.gz highway-1.4.0.tar.gz
https://github.com/google/libultrahdr/archive/v2.0.2.tar.gz libultrahdr-2.0.2.tar.gz
https://github.com/harfbuzz/harfbuzz/archive/14.5.0.tar.gz harfbuzz-14.5.0.tar.gz
https://github.com/libarchive/libarchive/releases/download/v3.8.9/libarchive-3.8.9.tar.xz libarchive-3.8.9.tar.xz
https://github.com/libexif/libexif/releases/download/v0.6.26/libexif-0.6.26.tar.xz libexif-0.6.26.tar.xz
https://github.com/libexpat/libexpat/releases/download/R_2_8_5/expat-2.8.5.tar.xz expat-2.8.5.tar.xz
https://github.com/libffi/libffi/releases/download/v3.8.0/libffi-3.8.0.tar.gz libffi-3.8.0.tar.gz
https://github.com/libvips/libvips/releases/download/v8.18.7/vips-8.18.7.tar.xz vips-8.18.7.tar.xz
https://github.com/lovell/libimagequant/archive/v2.4.1.tar.gz libimagequant-2.4.1.tar.gz
https://github.com/mm2/Little-CMS/releases/download/lcms2.19.1/lcms2-2.19.1.tar.gz lcms2-2.19.1.tar.gz
https://github.com/mozilla/mozjpeg/archive/0826579.tar.gz mozjpeg-0826579.tar.gz
https://github.com/mozilla/mozjpeg/commit/f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch mozjpeg-f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch
https://github.com/pnggroup/libpng/archive/v1.6.58.tar.gz libpng-1.6.58.tar.gz
https://github.com/strukturag/libheif/releases/download/v1.23.5/libheif-1.23.5.tar.gz libheif-1.23.5.tar.gz
https://github.com/zlib-ng/zlib-ng/archive/2.3.3.tar.gz zlib-ng-2.3.3.tar.gz
https://gitlab.com/libtiff/libtiff/-/archive/v4.7.2/libtiff-v4.7.2.tar.gz libtiff-v4.7.2.tar.gz
https://gitlab.freedesktop.org/fontconfig/fontconfig/-/archive/2.18.3/fontconfig-2.18.3.tar.gz fontconfig-2.18.3.tar.gz
https://patch-diff.githubusercontent.com/raw/google/libultrahdr/pull/383.patch libultrahdr-pull-383.patch
https://raw.githubusercontent.com/lovell/sharp-libvips/main/THIRD-PARTY-NOTICES.md THIRD-PARTY-NOTICES.md
https://storage.googleapis.com/aom-releases/libaom-3.15.1.tar.gz libaom-3.15.1.tar.gz
https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.6.0.tar.gz libwebp-1.6.0.tar.gz
1 https://cairographics.org/releases/cairo-1.18.6.tar.xz cairo-1.18.6.tar.xz
2 https://cairographics.org/releases/pixman-0.46.4.tar.gz pixman-0.46.4.tar.gz
3 https://download.gnome.org/sources/glib/2.90/glib-2.90.0.tar.xz glib-2.90.0.tar.xz
4 https://download.gnome.org/sources/librsvg/2.63/librsvg-2.63.2.tar.xz librsvg-2.63.2.tar.xz
5 https://download.gnome.org/sources/libxml2/2.15/libxml2-2.15.4.tar.xz libxml2-2.15.4.tar.xz
6 https://download.gnome.org/sources/pango/1.58/pango-1.58.2.tar.xz pango-1.58.2.tar.xz
7 https://gist.github.com/kleisauke/284d685efa00908da99ea6afbaaf39ae/raw/bdad5489a61c217850631571caf57f5db6ea8b2c/glib-without-gregex.patch glib-without-gregex.patch
8 https://gist.githubusercontent.com/lovell/313a6901e9db1bf285f2a1f1180499e4/raw/3988223c7dfa4d22745d9392034b0117abef1446/libvips-cpp-soversion.patch libvips-cpp-soversion.patch
9 https://github.com/GNOME/librsvg/commit/9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch librsvg-9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch
10 https://github.com/dloebl/cgif/archive/v0.5.4.tar.gz cgif-0.5.4.tar.gz
11 https://github.com/freetype/freetype/archive/VER-2-14-3.tar.gz freetype-2.14.3.tar.gz
12 https://github.com/fribidi/fribidi/releases/download/v1.0.17/fribidi-1.0.17.tar.xz fribidi-1.0.17.tar.xz
13 https://github.com/google/highway/archive/1.4.0.tar.gz highway-1.4.0.tar.gz
14 https://github.com/google/libultrahdr/archive/v2.0.2.tar.gz libultrahdr-2.0.2.tar.gz
15 https://github.com/harfbuzz/harfbuzz/archive/14.5.0.tar.gz harfbuzz-14.5.0.tar.gz
16 https://github.com/libarchive/libarchive/releases/download/v3.8.9/libarchive-3.8.9.tar.xz libarchive-3.8.9.tar.xz
17 https://github.com/libexif/libexif/releases/download/v0.6.26/libexif-0.6.26.tar.xz libexif-0.6.26.tar.xz
18 https://github.com/libexpat/libexpat/releases/download/R_2_8_5/expat-2.8.5.tar.xz expat-2.8.5.tar.xz
19 https://github.com/libffi/libffi/releases/download/v3.8.0/libffi-3.8.0.tar.gz libffi-3.8.0.tar.gz
20 https://github.com/libvips/libvips/releases/download/v8.18.7/vips-8.18.7.tar.xz vips-8.18.7.tar.xz
21 https://github.com/lovell/libimagequant/archive/v2.4.1.tar.gz libimagequant-2.4.1.tar.gz
22 https://github.com/mm2/Little-CMS/releases/download/lcms2.19.1/lcms2-2.19.1.tar.gz lcms2-2.19.1.tar.gz
23 https://github.com/mozilla/mozjpeg/archive/0826579.tar.gz mozjpeg-0826579.tar.gz
24 https://github.com/mozilla/mozjpeg/commit/f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch mozjpeg-f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch
25 https://github.com/pnggroup/libpng/archive/v1.6.58.tar.gz libpng-1.6.58.tar.gz
26 https://github.com/strukturag/libheif/releases/download/v1.23.5/libheif-1.23.5.tar.gz libheif-1.23.5.tar.gz
27 https://github.com/zlib-ng/zlib-ng/archive/2.3.3.tar.gz zlib-ng-2.3.3.tar.gz
28 https://gitlab.com/libtiff/libtiff/-/archive/v4.7.2/libtiff-v4.7.2.tar.gz libtiff-v4.7.2.tar.gz
29 https://gitlab.freedesktop.org/fontconfig/fontconfig/-/archive/2.18.3/fontconfig-2.18.3.tar.gz fontconfig-2.18.3.tar.gz
30 https://patch-diff.githubusercontent.com/raw/google/libultrahdr/pull/383.patch libultrahdr-pull-383.patch
31 https://raw.githubusercontent.com/lovell/sharp-libvips/main/THIRD-PARTY-NOTICES.md THIRD-PARTY-NOTICES.md
32 https://storage.googleapis.com/aom-releases/libaom-3.15.1.tar.gz libaom-3.15.1.tar.gz
33 https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.6.0.tar.gz libwebp-1.6.0.tar.gz
File diff suppressed because it is too large Load Diff
+2
View File
@@ -5,11 +5,13 @@
"type": "module",
"files": [
"bin",
"corresponding-sources",
"dist",
"license-supplements",
"licenses"
],
"bin": {
"ekklesia-corresponding-source": "./bin/ekklesia-corresponding-source.mjs",
"ekklesia-pack-module": "./bin/ekklesia-pack-module.mjs"
},
"exports": {
@@ -0,0 +1,155 @@
#!/usr/bin/env node
import path from "node:path";
import process from "node:process";
import { pathToFileURL } from "node:url";
import {
acquireCorrespondingSources,
buildCorrespondingSource,
CorrespondingSourceError,
verifyRedistribution,
} from "./corresponding-source.js";
/**
* `ekklesia-corresponding-source` — corresponding-source artifacts for native packages (ADR 0055).
*
* ekklesia-corresponding-source acquire (--package <module package> | --recipe <name>@<version>) --cache <directory>
* Downloads every recorded source file the package's covered native dependencies (or the named
* recipe) need into the verified cache. The only subcommand that uses the network.
* ekklesia-corresponding-source build --package <module package> --cache <directory> --out <directory>
* Offline. Writes <package>-corresponding-source.tar.gz, its .sha256, <package>.redistribution.json,
* and <package>.source-notice.txt.
* ekklesia-corresponding-source verify --package <module package> [--source <archive> --metadata <json>]
* The public-distribution gate. Exits 0 only when the package is approved.
*/
const USAGE = [
"Usage:",
" ekklesia-corresponding-source acquire (--package <archive> | --recipe <name>@<version>) --cache <directory>",
" ekklesia-corresponding-source build --package <archive> --cache <directory> --out <directory>",
" ekklesia-corresponding-source verify --package <archive> [--source <archive> --metadata <file>]",
].join("\n");
const OPTIONS: Readonly<Record<string, readonly string[]>> = Object.freeze({
acquire: ["--package", "--recipe", "--cache"],
build: ["--package", "--cache", "--out"],
verify: ["--package", "--source", "--metadata"],
});
const REQUIRED: Readonly<Record<string, readonly string[]>> = Object.freeze({
acquire: ["--cache"],
build: ["--package", "--cache", "--out"],
verify: ["--package"],
});
class ArgumentsError extends Error {}
export function parseCorrespondingSourceArguments(argv: readonly string[]): {
readonly command: "acquire" | "build" | "verify";
readonly values: ReadonlyMap<string, string>;
} {
const [command, ...rest] = argv;
const allowed = command === undefined ? undefined : OPTIONS[command];
if (allowed === undefined) {
throw new ArgumentsError(USAGE);
}
const values = new Map<string, string>();
for (let index = 0; index < rest.length; index += 2) {
const option = rest[index] ?? "";
const value = rest[index + 1];
if (
!allowed.includes(option) ||
value === undefined ||
value.startsWith("--") ||
values.has(option)
) {
throw new ArgumentsError(USAGE);
}
values.set(option, option === "--recipe" ? value : path.resolve(value));
}
if (command === "acquire" && values.has("--package") === values.has("--recipe")) {
throw new ArgumentsError(USAGE);
}
if ((REQUIRED[command as string] ?? []).some((option) => !values.has(option))) {
throw new ArgumentsError(USAGE);
}
return { command: command as "acquire" | "build" | "verify", values };
}
async function main(): Promise<number> {
try {
const { command, values } = parseCorrespondingSourceArguments(process.argv.slice(2));
const packageArchivePath = values.get("--package") ?? "";
if (command === "acquire") {
const recipe = values.get("--recipe");
const at = recipe?.lastIndexOf("@") ?? -1;
if (recipe !== undefined && at <= 0) {
throw new ArgumentsError(USAGE);
}
const result = await acquireCorrespondingSources({
cacheDirectory: values.get("--cache") ?? "",
...(recipe === undefined
? { packageArchivePath }
: { recipes: [{ name: recipe.slice(0, at), version: recipe.slice(at + 1) }] }),
});
process.stdout.write(
`Source cache holds all ${result.files} recorded files (${result.bytes} bytes); fetched ${result.fetched}.\n`,
);
} else if (command === "build") {
const result = await buildCorrespondingSource({
cacheDirectory: values.get("--cache") ?? "",
outDirectory: values.get("--out") ?? "",
packageArchivePath,
});
if (!result.required) {
process.stdout.write(
"The package redistributes no covered native code: no artifact is needed.\n",
);
} else {
process.stdout.write(
`Wrote ${path.basename(result.sourceArchivePath ?? "")}, ${result.sourceBytes} bytes\nsha256 ${result.sourceSha256}\n`,
);
}
} else {
const metadataPath = values.get("--metadata");
const sourceArchivePath = values.get("--source");
const approval = await verifyRedistribution({
packageArchivePath,
...(metadataPath === undefined ? {} : { metadataPath }),
...(sourceArchivePath === undefined ? {} : { sourceArchivePath }),
});
process.stdout.write(
approval.correspondingSource === undefined
? `Approved: ${approval.package.file} redistributes no covered native code.\n`
: `Approved: ${approval.package.file} with ${approval.correspondingSource.file} (sha256 ${approval.correspondingSource.sha256}).\n`,
);
}
return 0;
} catch (error) {
if (error instanceof ArgumentsError) {
process.stderr.write(`${error.message}\n`);
return 64;
}
if (error instanceof CorrespondingSourceError) {
process.stderr.write(
`${error.code}: ${error.message}${error.subject ? ` (${error.subject})` : ""}\n`,
);
return 1;
}
process.stderr.write("ekklesia-corresponding-source failed unexpectedly.\n");
return 1;
}
}
function isDirectExecution(): boolean {
const entrypoint = process.argv[1];
// Run through the committed launcher, which imports this file, or directly.
return (
entrypoint !== undefined &&
(import.meta.url === pathToFileURL(path.resolve(entrypoint)).href ||
path.basename(entrypoint) === "ekklesia-corresponding-source.mjs" ||
path.basename(entrypoint) === "ekklesia-corresponding-source")
);
}
if (isDirectExecution()) {
process.exitCode = await main();
}
File diff suppressed because it is too large Load Diff
+1
View File
@@ -1 +1,2 @@
export * from "./corresponding-source.js";
export * from "./pack-module.js";
+99 -1
View File
@@ -1,4 +1,4 @@
import { gzipSync } from "node:zlib";
import { gunzipSync, gzipSync } from "node:zlib";
/**
* The deterministic archive encoding of Ekklesia Module Package v1 (ADR 0049).
@@ -107,3 +107,101 @@ export function deterministicTarGz(topLevel: string, files: ReadonlyMap<string,
parts.push(Buffer.alloc(BLOCK * 2));
return gzipSync(Buffer.concat(parts), { level: GZIP_LEVEL });
}
/** One entry of a ustar archive, as read back. */
export interface TarEntry {
readonly path: string;
readonly directory: boolean;
readonly data: Buffer;
}
export class TarFormatError extends Error {
constructor() {
super("An archive is not in Ekklesia's deterministic ustar encoding.");
this.name = "TarFormatError";
}
}
function field(block: Buffer, offset: number, length: number): string {
const raw = block.subarray(offset, offset + length);
const end = raw.indexOf(0);
return raw.subarray(0, end === -1 ? length : end).toString("latin1");
}
/**
* Reads an archive written by `deterministicTarGz` back into its entries, refusing anything that
* encoding never produces: a link or special file, another header type, a bad checksum, or data
* after the end blocks. Whether the archive is byte-for-byte canonical is checked separately, by
* writing the entries again (`isCanonicalTarGz`).
*/
export function readTarGz(archive: Buffer): readonly TarEntry[] {
let raw: Buffer;
try {
raw = gunzipSync(archive);
} catch {
throw new TarFormatError();
}
const entries: TarEntry[] = [];
let offset = 0;
for (;;) {
if (offset + BLOCK > raw.length) {
throw new TarFormatError();
}
const block = raw.subarray(offset, offset + BLOCK);
if (block.every((byte) => byte === 0)) {
break;
}
const stored = Number.parseInt(field(block, 148, 8).trim() || "x", 8);
let checksum = 0;
for (let index = 0; index < BLOCK; index += 1) {
checksum += index >= 148 && index < 156 ? 32 : (block[index] ?? 0);
}
const typeflag = field(block, 156, 1);
const size = Number.parseInt(field(block, 124, 12) || "x", 8);
if (
stored !== checksum ||
(typeflag !== "0" && typeflag !== "5") ||
!Number.isSafeInteger(size)
) {
throw new TarFormatError();
}
const name = field(block, 0, 100);
const prefix = field(block, 345, 155);
const joined = prefix === "" ? name : `${prefix}/${name}`;
const directory = typeflag === "5";
const start = offset + BLOCK;
if (start + size > raw.length) {
throw new TarFormatError();
}
entries.push({
data: Buffer.from(raw.subarray(start, start + size)),
directory,
path: directory ? joined.replace(/\/$/u, "") : joined,
});
offset = start + Math.ceil(size / BLOCK) * BLOCK;
}
return entries;
}
/** Whether `archive` is exactly what `deterministicTarGz` writes for its own entries: the same
* order, headers, and padding. Compared uncompressed, so a different zlib cannot matter. */
export function isCanonicalTarGz(archive: Buffer, entries: readonly TarEntry[]): boolean {
const topLevel = entries[0];
if (topLevel === undefined || !topLevel.directory || topLevel.path.includes("/")) {
return false;
}
const files = new Map<string, Buffer>();
for (const entry of entries) {
if (!entry.directory) {
if (!entry.path.startsWith(`${topLevel.path}/`)) {
return false;
}
files.set(entry.path.slice(topLevel.path.length + 1), entry.data);
}
}
try {
return gunzipSync(deterministicTarGz(topLevel.path, files)).equals(gunzipSync(archive));
} catch {
return false;
}
}
@@ -0,0 +1,342 @@
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { existsSync } from "node:fs";
import { readFile, realpath, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { currentModulePackageHost, verifyModulePackage } from "@ekklesia/core/modules";
import { afterEach, describe, expect, it } from "vitest";
import {
buildCorrespondingSource,
CorrespondingSourceError,
loadSourceRecipes,
verifyRedistribution,
} from "../src/corresponding-source.js";
import { packModule } from "../src/index.js";
import { NATIVE_TARGETS, nativeClosure } from "../src/native.js";
import {
contentDigest,
DEFAULT_SUPPLEMENTS_DIRECTORY,
loadLicenseSupplements,
} from "../src/supplements.js";
import { readTarGz } from "../src/tar.js";
import {
cleanupFixtures,
installedPackageFile,
nativeFixtureModule,
SOURCE_REVISION,
scratchDirectory,
} from "./support/fixture-module.js";
/**
* ADR 0055's first complete proof: the native proof module with sharp 0.35.5, whose libvips 1.3.4
* binary contains LGPL and MPL components, and its real corresponding-source artifact.
*
* The recipe checks run everywhere. Building the artifact needs the recorded sources in a verified
* cache, which `pnpm test:corresponding-source` acquires once and passes in as
* EKKLESIA_CORRESPONDING_SOURCE_CACHE; without it, that part is skipped rather than downloading
* about 200 MB on every run.
*/
const execFileAsync = promisify(execFile);
const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const repositoryRoot = path.resolve(packageRoot, "..", "..");
const LIBVIPS = "@img/sharp-libvips-linux-x64";
const cache = process.env.EKKLESIA_CORRESPONDING_SOURCE_CACHE;
const host = currentModulePackageHost();
const onTargetHost =
host.platform === "linux" && host.arch === "x64" && host.libc?.family === "glibc";
afterEach(cleanupFixtures);
// `pnpm test:corresponding-source` asks for the full proof: then a missing cache is a failure, not a
// skip.
it.runIf(process.env.EKKLESIA_CORRESPONDING_SOURCE_REQUIRED === "1")(
"has the verified source cache it was asked to use, on a linux-x64-glibc host",
() => {
expect(onTargetHost).toBe(true);
expect(cache !== undefined && existsSync(cache)).toBe(true);
},
);
describe("the libvips 1.3.4 source recipe", () => {
it("loads intact, matches its reviewed supplement exactly, and pins the installed package", async () => {
const recipes = await loadSourceRecipes();
const [loaded] = recipes.get(LIBVIPS) ?? [];
expect(loaded?.recipe.package).toMatchObject({
license: "LGPL-3.0-or-later",
target: "linux-x64-glibc",
version: "1.3.4",
});
// Prepared against exactly the reviewed supplement for these package bytes.
const supplementJson = await readFile(
path.join(DEFAULT_SUPPLEMENTS_DIRECTORY, LIBVIPS, "1.3.4", "supplement.json"),
);
expect(loaded?.recipe.supplement.sha256).toBe(
createHash("sha256").update(supplementJson).digest("hex"),
);
const supplement = (await loadLicenseSupplements())
.get(LIBVIPS)
?.find((candidate) => candidate.version === "1.3.4");
expect(loaded?.recipe.package.contentDigest).toBe(supplement?.contentDigest);
// And those are the bytes sharp 0.35.5 installs.
const root = await realpath(await installedPackageFile("sharp"));
const target = NATIVE_TARGETS.get("linux-x64-glibc");
if (target === undefined) throw new Error("target");
const installed = (await nativeClosure([{ name: "sharp", root }], target)).find(
(entry) => entry.name === LIBVIPS,
);
expect(installed && contentDigest(installed.files)).toBe(loaded?.recipe.package.contentDigest);
// Every component the supplement records inside the binary maps to a source component. The one
// exception is the package's own entry, which carries the standard texts of its declared licence.
const referenced = new Set(
(loaded?.recipe.components ?? []).flatMap((component) =>
component.licenseMaterial.map((material) => `${material.name} ${material.version}`),
),
);
const unmapped = (supplement?.components ?? [])
.map((component) => `${component.name} ${component.version}`)
.filter((component) => !referenced.has(component));
expect(unmapped).toEqual([`${LIBVIPS} 1.3.4`]);
const components = loaded?.recipe.components ?? [];
const byRole = (role: string) =>
components.filter((component) => component.role === role).length;
expect({
buildDefinition: byRole("build-definition"),
buildInput: byRole("build-input"),
compiled: byRole("compiled"),
}).toEqual({
buildDefinition: 1,
buildInput: 206,
compiled: 176,
});
// The five patches build/posix.sh applies, and the Rust standard library of the recorded nightly.
expect(
components.flatMap((component) => component.patches.map((patch) => patch.file)).sort(),
).toEqual([
"patches/glib-without-gregex.patch",
"patches/librsvg-9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch",
"patches/libultrahdr-pull-383.patch",
"patches/libvips-cpp-soversion.patch",
"patches/mozjpeg-f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch",
]);
expect(
components.find((component) => component.name === "Rust standard library")?.sources[0]?.file,
).toBe("sources/rust/rust-src-nightly-2026-09-27.tar.xz");
// gvdb ships inside glib's archive; proxy-libintl is never built for this target.
expect(components.find((component) => component.name === "gvdb")?.includedIn).toEqual({
component: "glib",
path: "subprojects/gvdb",
});
expect(components.some((component) => component.name === "proxy-libintl")).toBe(false);
expect(loaded?.recipe.exclusions.map((exclusion) => exclusion.name)).toContain("proxy-libintl");
});
});
async function packNative(): Promise<string> {
const moduleDirectory = await nativeFixtureModule();
const outDirectory = await scratchDirectory("ekklesia-native-out-");
const result = await packModule({
licenseFile: path.join(repositoryRoot, "LICENSE"),
moduleDirectory,
nodeRange: ">=24.11.0 <25",
outDirectory,
sourceRevision: SOURCE_REVISION,
target: "linux-x64-glibc",
});
return result.archivePath;
}
describe.runIf(onTargetHost && cache !== undefined && existsSync(cache ?? ""))(
"the native proof's corresponding source, from the verified cache",
() => {
it("builds deterministically, is approved by the gate, and fails closed without it", {
timeout: 600_000,
}, async () => {
const first = await packNative();
const second = await packNative();
expect((await readFile(first)).equals(await readFile(second))).toBe(true);
const outA = await scratchDirectory("ekklesia-source-a-");
const outB = path.join(await scratchDirectory("ekklesia-unrelated-"), "nested", "out");
const a = await buildCorrespondingSource({
cacheDirectory: cache ?? "",
outDirectory: outA,
packageArchivePath: first,
});
const b = await buildCorrespondingSource({
cacheDirectory: cache ?? "",
outDirectory: outB,
packageArchivePath: second,
});
expect(a.required).toBe(true);
expect(a.sourceSha256).toBe(b.sourceSha256);
expect(
(await readFile(a.sourceArchivePath ?? "")).equals(
await readFile(b.sourceArchivePath ?? ""),
),
).toBe(true);
expect(path.basename(a.sourceArchivePath ?? "")).toBe(
"native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz",
);
// What it holds: every recorded source, as bytes, and nothing unrecorded.
const archive = await readFile(a.sourceArchivePath ?? "");
const entries = readTarGz(archive);
const top = "native-proof-0.1.0-linux-x64-glibc-corresponding-source";
const files = entries
.filter((entry) => !entry.directory)
.map((entry) => entry.path.slice(top.length + 1));
const manifest = JSON.parse(
entries
.find((entry) => entry.path === `${top}/corresponding-source.json`)
?.data.toString("utf8") ?? "{}",
);
const dependency = manifest.dependencies[0];
expect(dependency).toMatchObject({
name: LIBVIPS,
version: "1.3.4",
binaries: ["lib/libvips-cpp.so.8.18.7"],
});
const base = `dependencies/${LIBVIPS}/1.3.4`;
const sources = files.filter((file) => file.startsWith(`${base}/sources/`));
expect(sources.filter((file) => file.startsWith(`${base}/sources/crates/`))).toHaveLength(
353,
);
expect(
sources.filter((file) => /\/sources\/[^/]+\.tar\.(?:gz|xz)$/u.test(file)),
).toHaveLength(28);
expect(files.filter((file) => file.startsWith(`${base}/patches/`))).toHaveLength(5);
for (const required of [
"BUILDING.md",
"PROVENANCE.md",
"recipe.json",
"licenses/supplement.json",
"build/prepare-offline.sh",
"build/Cargo.lock",
"sources/rust/rust-src-nightly-2026-09-27.tar.xz",
]) {
expect(files, required).toContain(`${base}/${required}`);
}
expect(new Set(manifest.files.map((file: { path: string }) => file.path))).toEqual(
new Set(files.filter((file) => file !== "corresponding-source.json")),
);
// The binary-source relationship, by digest.
const metadata = JSON.parse(await readFile(a.metadataPath ?? "", "utf8"));
const packageSha256 = createHash("sha256")
.update(await readFile(first))
.digest("hex");
expect(metadata).toMatchObject({
correspondingSource: {
file: path.basename(a.sourceArchivePath ?? ""),
sha256: a.sourceSha256,
size: archive.length,
},
covered: [
{
contentDigest:
"ekklesia-content-v1:sha256:2927fe11493461fa34847191d0b9b1c6441007c3e99f306e5708cb07fb0a5e59",
name: LIBVIPS,
version: "1.3.4",
},
],
package: {
file: path.basename(first),
module: { id: "native-proof", version: "0.1.0" },
sha256: packageSha256,
target: "linux-x64-glibc",
},
});
expect(metadata.notice).toContain("GNU LGPL and MPL");
expect(metadata.notice).toContain(`SHA-256 ${a.sourceSha256}`);
// The gate approves exactly this, and fails closed when it is removed or corrupted.
await expect(
verifyRedistribution({
metadataPath: a.metadataPath ?? "",
packageArchivePath: first,
sourceArchivePath: a.sourceArchivePath ?? "",
}),
).resolves.toMatchObject({ approved: true, correspondingSource: { sha256: a.sourceSha256 } });
const missing = verifyRedistribution({
metadataPath: a.metadataPath ?? "",
packageArchivePath: first,
});
await expect(missing).rejects.toBeInstanceOf(CorrespondingSourceError);
await expect(missing).rejects.toMatchObject({
code: "CORRESPONDING_SOURCE_ARTIFACT_MISSING",
});
const corrupted = Buffer.from(archive);
const flip = Math.floor(corrupted.length / 3);
corrupted[flip] = (corrupted[flip] ?? 0) ^ 0x01;
await writeFile(b.sourceArchivePath ?? "", corrupted);
await expect(
verifyRedistribution({
metadataPath: b.metadataPath ?? "",
packageArchivePath: second,
sourceArchivePath: b.sourceArchivePath ?? "",
}),
).rejects.toMatchObject({
code: expect.stringMatching(
/^CORRESPONDING_SOURCE_(?:ARTIFACT_INVALID|MANIFEST_MISMATCH|METADATA_MISMATCH)$/u,
),
});
await rm(b.sourceArchivePath ?? "");
await expect(
verifyRedistribution({
metadataPath: b.metadataPath ?? "",
packageArchivePath: second,
sourceArchivePath: b.sourceArchivePath ?? "",
}),
).rejects.toMatchObject({ code: "CORRESPONDING_SOURCE_ARTIFACT_MISSING" });
});
it("leaves the binary package itself unchanged: it verifies, and sharp runs from it", {
timeout: 300_000,
}, async () => {
const archivePath = await packNative();
const archive = await readFile(archivePath);
const extractTo = path.join(
await scratchDirectory("ekklesia-native-installed-"),
"native-proof",
);
await expect(
verifyModulePackage({
archivePath,
expectedSha256: createHash("sha256").update(archive).digest("hex"),
extractTo,
platform: {
databaseRuntime: { "drizzle-orm": "0.45.2" },
moduleSdkVersion: "0.2.1",
platformVersion: "0.2.1",
},
}),
).resolves.toMatchObject({ moduleId: "native-proof", target: "linux-x64-glibc" });
// The source archive never travels inside the package.
expect(readTarGz(archive).some((entry) => /corresponding-source/u.test(entry.path))).toBe(
false,
);
const { stdout } = await execFileAsync(
process.execPath,
[
"--input-type=module",
"-e",
`const m = await import(${JSON.stringify(`file://${extractTo}/server/index.js`)}); process.stdout.write(JSON.stringify(await m.proof()));`,
],
{ cwd: tmpdir(), env: {} },
);
expect(JSON.parse(stdout)).toEqual({
pixels: [255, 0, 0, 255, 0, 0, 255, 0, 0, 255, 0, 0],
png: { format: "png", height: 3, width: 5 },
resized: { channels: 3, height: 2, width: 2 },
});
});
},
);
@@ -0,0 +1,774 @@
import { createHash } from "node:crypto";
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import path from "node:path";
import { gunzipSync, gzipSync } from "node:zlib";
import { afterEach, describe, expect, it } from "vitest";
import {
acquireCorrespondingSources,
buildCorrespondingSource,
CorrespondingSourceError,
type CorrespondingSourceErrorCode,
hasSourceObligations,
loadSourceRecipes,
verifyRedistribution,
} from "../src/corresponding-source.js";
import { contentDigest } from "../src/supplements.js";
import { deterministicTarGz, readTarGz } from "../src/tar.js";
import { cleanupFixtures, scratchDirectory } from "./support/fixture-module.js";
/**
* Corresponding-source artifacts and the public-distribution gate (ADR 0055), on small synthetic
* inputs: a fake LGPL native package, its reviewed supplement, its source recipe, and a source
* cache. The real libvips proof is `corresponding-source-native.test.ts`.
*/
afterEach(cleanupFixtures);
function sha256(data: string | Uint8Array): string {
return createHash("sha256").update(data).digest("hex");
}
const PACKAGE_FILE = "example-0.1.0-linux-x64-glibc.tar.gz";
const SOURCE_FILE = "example-0.1.0-linux-x64-glibc-corresponding-source.tar.gz";
const METADATA_FILE = "example-0.1.0-linux-x64-glibc.redistribution.json";
const TOP = "example-0.1.0-linux-x64-glibc-corresponding-source";
const SOURCES: Readonly<Record<string, string>> = {
"sources/fake-build-1.0.tar.gz": "build definitions for libfake\n",
"sources/fakelib-2.0.tar.gz": "fakelib 2.0 source\n",
"patches/fakelib-fix.patch": "--- a/x\n+++ b/x\n@@ -1 +1 @@\n-a\n+b\n",
"sources/helperlib-1.1.tar.gz": "helperlib 1.1 source\n",
"sources/codegen-0.3.crate": "codegen 0.3 build-only source\n",
};
interface FixtureOptions {
readonly version?: string;
readonly libraryBytes?: string;
readonly license?: string;
readonly notices?: string;
readonly recipe?: (recipe: Record<string, unknown>) => void;
readonly withRecipe?: boolean;
}
interface Fixture {
readonly root: string;
readonly packagePath: string;
readonly cache: string;
readonly recipesDirectory: string;
readonly supplementsDirectory: string;
}
function noticeEntry(name: string, version: string, licence: string, uses: string): string {
const rule = "-".repeat(80);
return `${rule}\n${name} ${version}\nLicense: ${licence}\nUsed in: ${uses}\n${rule}\n\n[LICENSE]\n\ntext\n`;
}
/** Writes a fake native package into a Module Package archive, its supplement, recipe, and cache. */
async function fixture(options: FixtureOptions = {}): Promise<Fixture> {
const version = options.version ?? "1.0.0";
const license = options.license ?? "LGPL-2.1-or-later";
const root = await scratchDirectory("ekklesia-corresponding-");
const nativeFiles = new Map<string, Buffer>([
["lib/libfake.so.1", Buffer.from(options.libraryBytes ?? `\u007fELF fake library ${version}`)],
["package.json", Buffer.from(JSON.stringify({ license, name: "fake-native", version }))],
]);
const digest = contentDigest(nativeFiles);
const packageFiles = new Map<string, Buffer>([
[
"ekklesia.package.json",
Buffer.from(
JSON.stringify({
module: { id: "example", version: "0.1.0" },
packageFormat: 1,
target: "linux-x64-glibc",
}),
),
],
[
"THIRD_PARTY_NOTICES",
Buffer.from(options.notices ?? noticeEntry("fake-native", version, license, "node_modules")),
],
...[...nativeFiles].map(([file, data]) => [`node_modules/fake-native/${file}`, data] as const),
]);
const packageDirectory = path.join(root, "package");
await mkdir(packageDirectory);
const packagePath = path.join(packageDirectory, PACKAGE_FILE);
await writeFile(packagePath, deterministicTarGz("example", packageFiles));
// The reviewed supplement for exactly these package bytes.
const supplementsDirectory = path.join(root, "supplements");
const supplementDirectory = path.join(supplementsDirectory, "fake-native", version);
await mkdir(path.join(supplementDirectory, "texts"), { recursive: true });
const material = async (content: string) => {
await writeFile(path.join(supplementDirectory, "texts", `${sha256(content)}.txt`), content);
return `texts/${sha256(content)}.txt`;
};
const supplement = {
supplementFormat: 1,
package: { name: "fake-native", version, license, contentDigest: digest },
statement: "Covers the libraries built into lib/libfake.so.1.",
components: [
{
name: "fakelib",
version: "2.0",
license: "LGPL-2.1-or-later",
source: "https://example.org/fakelib-2.0.tar.gz",
material: [
{ title: "COPYING", text: await material("GNU LESSER GENERAL PUBLIC LICENSE (fake)\n") },
],
},
{
name: "helperlib",
version: "1.1",
license: "MIT",
source: "https://example.org/helperlib-1.1.tar.gz",
material: [{ title: "LICENSE", text: await material("MIT License (fake)\n") }],
},
],
};
const supplementJson = `${JSON.stringify(supplement, null, 2)}\n`;
await writeFile(path.join(supplementDirectory, "supplement.json"), supplementJson);
await writeFile(path.join(supplementDirectory, "PROVENANCE.md"), "# How this was assembled\n");
// The recipe, and a cache holding the recorded bytes.
const recipesDirectory = path.join(root, "recipes");
const cache = path.join(root, "cache");
await mkdir(path.join(cache, "sha256"), { recursive: true });
const file = async (relative: string) => {
const content = SOURCES[relative] ?? "";
await writeFile(path.join(cache, "sha256", sha256(content)), content);
return {
file: relative,
origin: `https://example.org/${path.basename(relative)}`,
sha256: sha256(content),
size: Buffer.byteLength(content),
};
};
const building = "# Rebuilding libfake\n\nRun `make` in fake-build.\n";
const recipe: Record<string, unknown> = {
recipeFormat: 1,
package: {
name: "fake-native",
version,
license,
contentDigest: digest,
target: "linux-x64-glibc",
binaries: ["lib/libfake.so.1"],
},
supplement: { version, sha256: sha256(supplementJson) },
documents: [
{ path: "BUILDING.md", sha256: sha256(building), size: Buffer.byteLength(building) },
],
components: [
{
name: "fake-build",
version: "1.0",
role: "build-definition",
license: "MIT",
relationship: "The build definitions.",
licenseMaterial: [],
sources: [await file("sources/fake-build-1.0.tar.gz")],
patches: [],
},
{
name: "fakelib",
version: "2.0",
role: "compiled",
license: "LGPL-2.1-or-later",
relationship: "Compiled into lib/libfake.so.1.",
licenseMaterial: [{ name: "fakelib", version: "2.0" }],
sources: [await file("sources/fakelib-2.0.tar.gz")],
patches: [await file("patches/fakelib-fix.patch")],
},
{
name: "helperlib",
version: "1.1",
role: "compiled",
license: "MIT",
relationship: "Compiled into lib/libfake.so.1.",
licenseMaterial: [{ name: "helperlib", version: "1.1" }],
sources: [await file("sources/helperlib-1.1.tar.gz")],
patches: [],
},
{
name: "codegen",
version: "0.3",
role: "build-input",
license: "MIT",
relationship: "Needed only to build.",
licenseMaterial: [],
sources: [await file("sources/codegen-0.3.crate")],
patches: [],
},
],
exclusions: [{ name: "testlib", reason: "Used only by tests." }],
};
options.recipe?.(recipe);
if (options.withRecipe !== false) {
const recipeDirectory = path.join(recipesDirectory, "fake-native", version);
await mkdir(recipeDirectory, { recursive: true });
await writeFile(
path.join(recipeDirectory, "recipe.json"),
`${JSON.stringify(recipe, null, 2)}\n`,
);
await writeFile(path.join(recipeDirectory, "BUILDING.md"), building);
}
return { cache, packagePath, recipesDirectory, root, supplementsDirectory };
}
async function build(fx: Fixture, out = path.join(fx.root, "out")) {
return buildCorrespondingSource({
cacheDirectory: fx.cache,
outDirectory: out,
packageArchivePath: fx.packagePath,
recipesDirectory: fx.recipesDirectory,
supplementsDirectory: fx.supplementsDirectory,
});
}
function verify(fx: Fixture, source?: string, metadata?: string) {
return verifyRedistribution({
packageArchivePath: fx.packagePath,
recipesDirectory: fx.recipesDirectory,
supplementsDirectory: fx.supplementsDirectory,
...(source === undefined ? {} : { sourceArchivePath: source }),
...(metadata === undefined ? {} : { metadataPath: metadata }),
});
}
async function rejects(promise: Promise<unknown>, code: CorrespondingSourceErrorCode) {
const error = await promise.then(
() => undefined,
(caught: unknown) => caught,
);
expect(error).toBeInstanceOf(CorrespondingSourceError);
expect((error as CorrespondingSourceError).code).toBe(code);
return error as CorrespondingSourceError;
}
/** A built release, ready to tamper with: its source archive's files, and a way to write a changed
* archive (and metadata naming it) where the gate will look. */
async function release() {
const fx = await fixture();
const out = path.join(fx.root, "out");
await build(fx, out);
const sourcePath = path.join(out, SOURCE_FILE);
const metadataPath = path.join(out, METADATA_FILE);
const files = new Map<string, Buffer>();
for (const entry of readTarGz(await readFile(sourcePath))) {
if (!entry.directory) files.set(entry.path.slice(TOP.length + 1), entry.data);
}
/** Writes `changed` as the archive and keeps the metadata consistent with its bytes, so only the
* change itself can fail the gate. */
const write = async (changed: Map<string, Buffer>, archive?: Buffer) => {
const bytes = archive ?? deterministicTarGz(TOP, changed);
await writeFile(sourcePath, bytes);
const metadata = JSON.parse(await readFile(metadataPath, "utf8"));
const digest = sha256(bytes);
metadata.correspondingSource.sha256 = digest;
metadata.correspondingSource.size = bytes.length;
metadata.correspondingSource.manifestSha256 = sha256(
changed.get("corresponding-source.json") ?? "",
);
metadata.notice = metadata.notice.replace(/SHA-256 [0-9a-f]{64}/u, `SHA-256 ${digest}`);
await writeFile(metadataPath, `${JSON.stringify(metadata, null, 2)}\n`);
};
/** Rewrites the manifest's file list to match `changed`, as a careful forger would. */
const relist = (changed: Map<string, Buffer>) => {
const manifest = JSON.parse(changed.get("corresponding-source.json")?.toString("utf8") ?? "{}");
manifest.files = [...changed]
.filter(([file]) => file !== "corresponding-source.json")
.map(([file, data]) => ({ path: file, size: data.length, sha256: sha256(data) }))
.sort((left: { path: string }, right: { path: string }) => (left.path < right.path ? -1 : 1));
changed.set("corresponding-source.json", Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`));
return manifest;
};
return { files, fx, metadataPath, out, relist, sourcePath, write };
}
describe("building the corresponding-source artifact", () => {
it("writes a deterministic archive, its digest, the redistribution metadata, and the release notice", async () => {
const fx = await fixture();
const first = await build(fx, path.join(fx.root, "out-a"));
// A second build into an unrelated directory, from a moved cache and package copy.
const second = await build(
fx,
path.join(await scratchDirectory("ekklesia-elsewhere-"), "out-b"),
);
expect(first.required).toBe(true);
expect(first.sourceSha256).toBe(second.sourceSha256);
expect(await readFile(first.sourceArchivePath ?? "")).toEqual(
await readFile(second.sourceArchivePath ?? ""),
);
expect(await readFile(`${first.sourceArchivePath}.sha256`, "utf8")).toBe(
`${first.sourceSha256} ${SOURCE_FILE}\n`,
);
const entries = readTarGz(await readFile(first.sourceArchivePath ?? ""));
const files = entries
.filter((entry) => !entry.directory)
.map((entry) => entry.path.slice(TOP.length + 1));
expect(files).toEqual([
"README.md",
"SHA256SUMS",
"corresponding-source.json",
"dependencies/fake-native/1.0.0/BUILDING.md",
"dependencies/fake-native/1.0.0/PROVENANCE.md",
"dependencies/fake-native/1.0.0/licenses/supplement.json",
expect.stringMatching(
/^dependencies\/fake-native\/1\.0\.0\/licenses\/texts\/[0-9a-f]{64}\.txt$/u,
),
expect.stringMatching(
/^dependencies\/fake-native\/1\.0\.0\/licenses\/texts\/[0-9a-f]{64}\.txt$/u,
),
"dependencies/fake-native/1.0.0/patches/fakelib-fix.patch",
"dependencies/fake-native/1.0.0/recipe.json",
"dependencies/fake-native/1.0.0/sources/codegen-0.3.crate",
"dependencies/fake-native/1.0.0/sources/fake-build-1.0.tar.gz",
"dependencies/fake-native/1.0.0/sources/fakelib-2.0.tar.gz",
"dependencies/fake-native/1.0.0/sources/helperlib-1.1.tar.gz",
]);
// The actual bytes, not references to them.
const patch = entries.find((entry) => entry.path.endsWith("patches/fakelib-fix.patch"));
expect(patch?.data.toString("utf8")).toBe(SOURCES["patches/fakelib-fix.patch"]);
const manifest = JSON.parse(
entries
.find((entry) => entry.path.endsWith("/corresponding-source.json"))
?.data.toString("utf8") ?? "{}",
);
expect(manifest.module).toEqual({ id: "example", target: "linux-x64-glibc", version: "0.1.0" });
expect(manifest.dependencies[0]).toMatchObject({
binaries: ["lib/libfake.so.1"],
exclusions: [{ name: "testlib", reason: "Used only by tests." }],
name: "fake-native",
version: "1.0.0",
});
expect(
manifest.dependencies[0].components.map((component: { role: string }) => component.role),
).toEqual(["build-definition", "compiled", "compiled", "build-input"]);
const metadata = JSON.parse(await readFile(first.metadataPath ?? "", "utf8"));
expect(metadata).toMatchObject({
correspondingSource: {
file: SOURCE_FILE,
sha256: first.sourceSha256,
manifestSha256: first.manifestSha256,
},
covered: [{ name: "fake-native", version: "1.0.0" }],
package: { file: PACKAGE_FILE, sha256: sha256(await readFile(fx.packagePath)) },
});
const notice = await readFile(first.noticePath ?? "", "utf8");
expect(notice).toBe(metadata.notice);
expect(notice).toContain(
"This release includes third-party software covered by licenses including the GNU LGPL",
);
expect(notice).toContain(
"Corresponding source and build/relink materials for the covered native dependencies",
);
expect(notice).toContain(SOURCE_FILE);
expect(notice).toContain(`SHA-256 ${first.sourceSha256}`);
// Nothing machine-specific in what the build generated.
for (const generated of ["README.md", "SHA256SUMS", "corresponding-source.json"]) {
const content =
entries.find((entry) => entry.path === `${TOP}/${generated}`)?.data.toString("utf8") ?? "";
expect(content).not.toContain(fx.root);
expect(content).not.toContain(fx.cache);
}
expect(JSON.stringify(metadata)).not.toContain(fx.root);
});
it("is approved by the gate exactly as built", async () => {
const { fx, metadataPath, sourcePath } = await release();
await expect(verify(fx, sourcePath, metadataPath)).resolves.toMatchObject({
approved: true,
correspondingSource: { file: SOURCE_FILE },
covered: [{ name: "fake-native", version: "1.0.0" }],
});
});
it("refuses to build from cached bytes that differ from the recipe, and writes nothing", async () => {
const fx = await fixture();
await writeFile(
path.join(fx.cache, "sha256", sha256(SOURCES["patches/fakelib-fix.patch"] ?? "")),
"changed patch\n",
);
await rejects(build(fx), "CORRESPONDING_SOURCE_SOURCE_MISMATCH");
await rm(path.join(fx.cache, "sha256", sha256(SOURCES["sources/fakelib-2.0.tar.gz"] ?? "")));
await expect(readFile(path.join(fx.root, "out", SOURCE_FILE))).rejects.toThrow();
});
it("refuses a missing cached file, and never overwrites an existing release", async () => {
const fx = await fixture();
await rm(path.join(fx.cache, "sha256", sha256(SOURCES["sources/helperlib-1.1.tar.gz"] ?? "")));
await rejects(build(fx), "CORRESPONDING_SOURCE_SOURCE_MISSING");
const fresh = await fixture();
await build(fresh);
await rejects(build(fresh), "CORRESPONDING_SOURCE_OUTPUT_EXISTS");
});
it("builds nothing for a package with no covered native code, and the gate approves it without an artifact", async () => {
const fx = await fixture({ license: "MIT", withRecipe: false });
// The fixture's supplement records an LGPL component, so remove it: a package whose own files and
// licence carry no source obligation needs no corresponding source.
await rm(fx.supplementsDirectory, { force: true, recursive: true });
await expect(build(fx)).resolves.toEqual({ required: false });
await expect(verify(fx)).resolves.toMatchObject({ approved: true, covered: [] });
});
it("treats a dependency as covered when its reviewed supplement records a copyleft component", async () => {
// Declared MIT, but the supplement says an LGPL library is built into it.
const fx = await fixture({ license: "MIT", withRecipe: false });
// The supplement's declared licence must match the package's for the supplement to apply.
const supplementPath = path.join(
fx.supplementsDirectory,
"fake-native",
"1.0.0",
"supplement.json",
);
const supplement = JSON.parse(await readFile(supplementPath, "utf8"));
expect(supplement.package.license).toBe("MIT");
await rejects(build(fx), "CORRESPONDING_SOURCE_RECIPE_MISSING");
});
});
describe("the public-distribution gate fails closed", () => {
it("without the corresponding-source archive, or without the redistribution metadata", async () => {
const { fx, metadataPath, sourcePath } = await release();
await rejects(verify(fx), "CORRESPONDING_SOURCE_ARTIFACT_MISSING");
await rm(sourcePath);
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_ARTIFACT_MISSING");
const second = await release();
await rejects(verify(second.fx, second.sourcePath), "CORRESPONDING_SOURCE_METADATA_MISMATCH");
});
it("when the metadata names a different source digest, or the archive's bytes are corrupted", async () => {
const { fx, metadataPath, sourcePath } = await release();
const metadata = JSON.parse(await readFile(metadataPath, "utf8"));
metadata.correspondingSource.sha256 = "0".repeat(64);
await writeFile(metadataPath, JSON.stringify(metadata));
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_METADATA_MISMATCH");
const corrupt = await release();
const bytes = await readFile(corrupt.sourcePath);
const middle = Math.floor(bytes.length / 2);
bytes[middle] = (bytes[middle] ?? 0) ^ 0xff;
await writeFile(corrupt.sourcePath, bytes);
await rejects(
verify(corrupt.fx, corrupt.sourcePath, corrupt.metadataPath),
"CORRESPONDING_SOURCE_ARTIFACT_INVALID",
);
});
it("when the release notice does not name the exact artifact", async () => {
const { fx, metadataPath, sourcePath } = await release();
const metadata = JSON.parse(await readFile(metadataPath, "utf8"));
metadata.notice = metadata.notice.replace(SOURCE_FILE, "some-other-source.tar.gz");
await writeFile(metadataPath, JSON.stringify(metadata));
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_METADATA_MISMATCH");
});
it("when the native package's bytes are not those the recipe was prepared for", async () => {
const built = await release();
// The same name and version, different library bytes: a different content digest.
const other = await fixture({ libraryBytes: "\u007fELF a different build" });
await writeFile(built.fx.packagePath, await readFile(other.packagePath));
await rejects(
verify(built.fx, built.sourcePath, built.metadataPath),
"CORRESPONDING_SOURCE_RECIPE_MISMATCH",
);
});
it("when a source component's version differs from the recipe", async () => {
const { files, fx, metadataPath, relist, sourcePath, write } = await release();
const manifest = JSON.parse(files.get("corresponding-source.json")?.toString("utf8") ?? "{}");
manifest.dependencies[0].components[1].version = "2.1";
files.set("corresponding-source.json", Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`));
relist(files);
await write(files);
const error = await rejects(
verify(fx, sourcePath, metadataPath),
"CORRESPONDING_SOURCE_MANIFEST_MISMATCH",
);
expect(error.subject).toBe("[email protected]");
});
it("when a required source component is missing", async () => {
const { files, fx, metadataPath, relist, sourcePath, write } = await release();
const manifest = JSON.parse(files.get("corresponding-source.json")?.toString("utf8") ?? "{}");
manifest.dependencies[0].components.splice(2, 1);
files.set("corresponding-source.json", Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`));
files.delete("dependencies/fake-native/1.0.0/sources/helperlib-1.1.tar.gz");
relist(files);
await write(files);
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_MANIFEST_MISMATCH");
});
it("when a patch's bytes changed, even with the file list rewritten to match", async () => {
const { files, fx, metadataPath, relist, sourcePath, write } = await release();
files.set(
"dependencies/fake-native/1.0.0/patches/fakelib-fix.patch",
Buffer.from("a different patch\n"),
);
relist(files);
await write(files);
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_MANIFEST_MISMATCH");
});
it("when the manifest records a file the archive does not hold", async () => {
const { files, fx, metadataPath, sourcePath, write } = await release();
files.delete("dependencies/fake-native/1.0.0/sources/fakelib-2.0.tar.gz");
await write(files);
const error = await rejects(
verify(fx, sourcePath, metadataPath),
"CORRESPONDING_SOURCE_MANIFEST_MISMATCH",
);
expect(error.subject).toBe("dependencies/fake-native/1.0.0/sources/fakelib-2.0.tar.gz");
});
it("when the archive holds source material its manifest does not record", async () => {
const { files, fx, metadataPath, sourcePath, write } = await release();
files.set(
"dependencies/fake-native/1.0.0/sources/unreviewed.tar.gz",
Buffer.from("not recorded\n"),
);
await write(files);
const error = await rejects(
verify(fx, sourcePath, metadataPath),
"CORRESPONDING_SOURCE_UNRECORDED_FILE",
);
expect(error.subject).toBe("dependencies/fake-native/1.0.0/sources/unreviewed.tar.gz");
});
it("when a generated file names a machine-specific path", async () => {
const { files, fx, metadataPath, relist, sourcePath, write } = await release();
files.set("README.md", Buffer.from("Built from /home/builder/cache/sha256.\n"));
relist(files);
await write(files);
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_LOCAL_PATH");
});
it("when the archive's entries are not in the deterministic order", async () => {
const { files, fx, metadataPath, sourcePath, write } = await release();
// The same entries and bytes, with the last two file entries swapped.
const raw = gunzipSync(await readFile(sourcePath));
const chunks: Buffer[] = [];
let offset = 0;
while (offset < raw.length && !raw.subarray(offset, offset + 512).every((byte) => byte === 0)) {
const size = Number.parseInt(
raw
.subarray(offset + 124, offset + 136)
.toString("latin1")
.replace(/\0.*$/u, "")
.trim() || "0",
8,
);
const length = 512 + Math.ceil(size / 512) * 512;
chunks.push(raw.subarray(offset, offset + length));
offset += length;
}
const last = chunks.length - 1;
[chunks[last - 1], chunks[last]] = [chunks[last] as Buffer, chunks[last - 1] as Buffer];
await write(files, gzipSync(Buffer.concat([...chunks, Buffer.alloc(1024)]), { level: 9 }));
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_ARTIFACT_INVALID");
});
it("when the artifact was built for another version of the native dependency", async () => {
// The 1.3.3-for-1.3.4 case: an artifact for fake-native 1.0.0 beside a package with 1.0.1.
const old = await release();
const current = await fixture({ version: "1.0.1" });
await mkdir(path.join(current.recipesDirectory, "fake-native", "1.0.0"), { recursive: true });
for (const file of ["recipe.json", "BUILDING.md"]) {
await writeFile(
path.join(current.recipesDirectory, "fake-native", "1.0.0", file),
await readFile(path.join(old.fx.recipesDirectory, "fake-native", "1.0.0", file)),
);
}
await rejects(
verify(current, old.sourcePath, old.metadataPath),
"CORRESPONDING_SOURCE_MANIFEST_MISMATCH",
);
});
it("when the package bundles copyleft code, which this mechanism does not cover", async () => {
const notices =
noticeEntry("fake-native", "1.0.0", "LGPL-2.1-or-later", "node_modules") +
noticeEntry("bundled-lgpl", "3.0.0", "LGPL-3.0-only", "backend");
const fx = await fixture({ notices });
await rejects(build(fx), "CORRESPONDING_SOURCE_BUNDLED_COPYLEFT");
await rejects(verify(fx), "CORRESPONDING_SOURCE_BUNDLED_COPYLEFT");
});
it("when a covered dependency has no recipe, or its recipe was prepared against another supplement", async () => {
await rejects(
build(await fixture({ withRecipe: false })),
"CORRESPONDING_SOURCE_RECIPE_MISSING",
);
const fx = await fixture({
recipe: (recipe) => {
(recipe.supplement as { sha256: string }).sha256 = "f".repeat(64);
},
});
await rejects(build(fx), "CORRESPONDING_SOURCE_RECIPE_MISMATCH");
const unknownMaterial = await fixture({
recipe: (recipe) => {
const components = recipe.components as {
licenseMaterial: { name: string; version: string }[];
}[];
(components[1] as { licenseMaterial: unknown[] }).licenseMaterial = [
{ name: "fakelib", version: "9.9" },
];
},
});
await rejects(build(unknownMaterial), "CORRESPONDING_SOURCE_RECIPE_MISMATCH");
});
});
describe("source recipes are reviewed inputs", () => {
async function loadWith(change: (recipe: Record<string, unknown>) => void) {
const fx = await fixture({ recipe: change });
return loadSourceRecipes(fx.recipesDirectory);
}
it("refuses an absolute or climbing source path, a non-https origin, or a compiled component without licence material", async () => {
const component = (recipe: Record<string, unknown>, index: number) =>
(recipe.components as Record<string, unknown>[])[index] as Record<string, unknown>;
const first = (recipe: Record<string, unknown>) =>
(component(recipe, 1).sources as Record<string, unknown>[])[0] as Record<string, unknown>;
for (const change of [
(recipe: Record<string, unknown>) => {
first(recipe).file = "/home/builder/fakelib-2.0.tar.gz";
},
(recipe: Record<string, unknown>) => {
first(recipe).file = "sources/../../fakelib-2.0.tar.gz";
},
(recipe: Record<string, unknown>) => {
first(recipe).origin = "http://example.org/fakelib-2.0.tar.gz";
},
(recipe: Record<string, unknown>) => {
component(recipe, 1).licenseMaterial = [];
},
(recipe: Record<string, unknown>) => {
component(recipe, 2).version = "2.0";
component(recipe, 2).name = "fakelib";
},
]) {
await rejects(loadWith(change), "CORRESPONDING_SOURCE_RECIPE_INVALID");
}
});
it("refuses a document whose bytes changed, and an unlisted file beside the recipe", async () => {
const fx = await fixture();
const directory = path.join(fx.recipesDirectory, "fake-native", "1.0.0");
await writeFile(path.join(directory, "BUILDING.md"), "edited without review\n");
await rejects(loadSourceRecipes(fx.recipesDirectory), "CORRESPONDING_SOURCE_RECIPE_INVALID");
const second = await fixture();
await writeFile(
path.join(second.recipesDirectory, "fake-native", "1.0.0", "notes.txt"),
"stray\n",
);
await rejects(
loadSourceRecipes(second.recipesDirectory),
"CORRESPONDING_SOURCE_RECIPE_INVALID",
);
});
it("recognizes licences with source obligations, by SPDX identifier or by upstream name", () => {
for (const licence of [
"LGPL-3.0-or-later",
"MPL-2.0",
"GPL-2.0-only",
"Mozilla Public License 1.1",
"MIT AND LGPL-2.1-or-later",
"EPL-2.0",
]) {
expect(hasSourceObligations(licence), licence).toBe(true);
}
for (const licence of [
"MIT",
"Apache-2.0",
"BSD-3-Clause",
"ISC",
"Zlib",
"MIT OR Apache-2.0",
]) {
expect(hasSourceObligations(licence), licence).toBe(false);
}
});
});
describe("acquisition is separate, verified, and the only network step", () => {
it("fetches each recorded file once, verifies it, and refuses a response that differs", async () => {
const fx = await fixture();
const fetched: string[] = [];
const served = new Map(
Object.entries(SOURCES).map(([file, content]) => [
`https://example.org/${path.basename(file)}`,
content,
]),
);
const empty = path.join(fx.root, "fresh-cache");
const result = await acquireCorrespondingSources({
cacheDirectory: empty,
fetch: async (url) => {
fetched.push(url);
return Buffer.from(served.get(url) ?? "");
},
packageArchivePath: fx.packagePath,
recipesDirectory: fx.recipesDirectory,
supplementsDirectory: fx.supplementsDirectory,
});
expect(result).toMatchObject({ fetched: 5, files: 5 });
// A second run finds everything verified and fetches nothing.
const again = await acquireCorrespondingSources({
cacheDirectory: empty,
fetch: async () => {
throw new Error("no network expected");
},
recipes: [{ name: "fake-native", version: "1.0.0" }],
recipesDirectory: fx.recipesDirectory,
});
expect(again).toMatchObject({ fetched: 0, files: 5 });
// The built artifact from the acquired cache is identical to one built from the original cache.
const fromAcquired = await buildCorrespondingSource({
cacheDirectory: empty,
outDirectory: path.join(fx.root, "acquired-out"),
packageArchivePath: fx.packagePath,
recipesDirectory: fx.recipesDirectory,
supplementsDirectory: fx.supplementsDirectory,
});
expect(fromAcquired.sourceSha256).toBe((await build(fx)).sourceSha256);
// A transient upstream failure is retried; only the recorded bytes are ever stored.
const flaky = path.join(fx.root, "flaky-cache");
const attempts = new Map<string, number>();
await expect(
acquireCorrespondingSources({
cacheDirectory: flaky,
fetch: async (url) => {
const count = (attempts.get(url) ?? 0) + 1;
attempts.set(url, count);
if (count === 1) {
throw new Error("connection reset");
}
return Buffer.from(served.get(url) ?? "");
},
recipes: [{ name: "fake-native", version: "1.0.0" }],
recipesDirectory: fx.recipesDirectory,
}),
).resolves.toMatchObject({ fetched: 5 });
const tampered = path.join(fx.root, "tampered-cache");
await rejects(
acquireCorrespondingSources({
cacheDirectory: tampered,
fetch: async (url) => Buffer.from(`${served.get(url) ?? ""}upstream changed\n`),
recipes: [{ name: "fake-native", version: "1.0.0" }],
recipesDirectory: fx.recipesDirectory,
}),
"CORRESPONDING_SOURCE_ACQUISITION_FAILED",
);
const { readdir } = await import("node:fs/promises");
expect(await readdir(path.join(tampered, "sha256"))).toEqual([]);
});
});
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# ADR 0055's full proof, with real sources: the native proof module (sharp 0.35.5, libvips 1.3.4) and
# its corresponding-source artifact.
#
# bash scripts/test-corresponding-source.sh
#
# 1. Acquisition (network, first run only): every checked-in recipe's recorded source files are
# downloaded into a verified cache, each checked against its recorded size and SHA-256. A file
# already in the cache with the right bytes is never fetched again.
# 2. Construction and the gate (offline): the artifact is built twice from unrelated directories and
# must be byte-identical; the public-distribution gate must approve it, and must refuse it once it
# is removed or corrupted; sharp must still run from the verified binary package.
#
# EKKLESIA_CORRESPONDING_SOURCE_CACHE chooses the cache directory; the default is
# ${XDG_CACHE_HOME:-$HOME/.cache}/ekklesia/corresponding-sources. It is about 200 MB per recipe.
set -Eeuo pipefail
root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$root"
cache="${EKKLESIA_CORRESPONDING_SOURCE_CACHE:-${XDG_CACHE_HOME:-$HOME/.cache}/ekklesia/corresponding-sources}"
pnpm exec turbo run build --filter=@ekklesia/module-pack... --filter=@ekklesia/core... >/dev/null
recipes="packages/module-pack/corresponding-sources"
while IFS= read -r recipe; do
relative="${recipe#"$recipes"/}"
relative="${relative%/recipe.json}"
echo "==> Acquiring the recorded sources for ${relative%/*}@${relative##*/}"
node packages/module-pack/bin/ekklesia-corresponding-source.mjs acquire \
--recipe "${relative%/*}@${relative##*/}" --cache "$cache"
done < <(find "$recipes" -name recipe.json | LC_ALL=C sort)
echo "==> Building, gating, and running the native proof offline"
EKKLESIA_CORRESPONDING_SOURCE_CACHE="$cache" EKKLESIA_CORRESPONDING_SOURCE_REQUIRED=1 \
pnpm --filter @ekklesia/module-pack exec vitest run tests/corresponding-source-native.test.ts