feat: corresponding-source artifacts for native packages that redistribute copyleft code (ADR 0055)

Ekklesia may publicly distribute a native Module Package containing LGPL or MPL components only beside a version-specific corresponding-source artifact. The artifact carries the exact source, patches, build definitions, and rebuild and relink instructions for those components. ADR 0055 (Accepted) records the policy: project compliance policy, not legal certification. It decides the item ADR 0052 deferred, and extends ADR 0049's distributable-release gate. Module Package v1, packageFormat 1, and both libvips supplements are unchanged. The mechanism is generic and lives in @ekklesia/module-pack: - Source recipes: reviewed data under corresponding-sources/<package>/<version>/. recipe.json names the exact package (name, version, content digest) and the exact supplement it was prepared with, and lists every component with its role, licence, licence material, and relationship to the binary. Every source file and patch is recorded by origin, size, and SHA-256; documents are hashed too, and every exclusion has a reason. Loading is strict and refuses anything unlisted. - Acquisition (the only network step) fetches each recorded file once into a cache keyed by SHA-256. It uses node:https with exact headers and no decoding, retries transient failures, and stores only bytes that match. - Construction is offline. From a built package it finds the covered native dependencies (by declared licence, or by a supplement component's licence), verifies every byte, and writes four sibling artifacts: the deterministic <package>-corresponding-source.tar.gz and its .sha256; <package>.redistribution.json, linking package, dependencies, supplement, recipe, and source by digest; and <package>.source-notice.txt, the release notice. - The public-distribution gate (verifyRedistribution, `ekklesia-corresponding-source verify`) approves only when the recipe, supplement, canonical archive, manifest, and metadata identify exactly these files. It refuses bundled copyleft code, and approves packages with no covered native code without an artifact. packModule is untouched. First complete proof: @img/[email protected] (sharp 0.35.5), from the reviewed supplement and provenance. - 383 components and 387 source files, about 208 MB: the sharp-libvips v1.3.4 build definitions, the 27 library archives, the five build-time patches, the 353 crates librsvg's resolved Cargo.lock records, and rust-src for nightly-2026-09-27 (rustc 75a75c3e0), which vendors std's dependencies. - BUILDING.md and two helpers point sharp-libvips's unchanged build logic at those files. - Followed in a Rocky Linux 8 container with networking disabled, they rebuilt libvips-cpp.so.8.18.7, byte-identically across two builds. Swapped into a verified, extracted native package, the rebuilt library was the one loaded, and sharp ran. - The native proof's source artifact, native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz, is 207242785 bytes, sha256 cc405492032b04a5391ab84e64f0e5e50f78e571ef7818204782c25a03eb50b7. It is byte-identical from unrelated directories and from a cache acquired fresh from upstream. Tests: - corresponding-source.test.ts runs in CI: synthetic inputs, every negative case, acquisition with retries and refusal, and determinism. - corresponding-source-native.test.ts checks the real recipe everywhere. With a verified cache (`pnpm test:corresponding-source`, which acquires once) it builds the real artifact twice, gates it, fails it closed, and runs sharp from the package. Docs: the module guide's distribution section, the README and architecture notes, and an ADR 0052 cross-reference. No Signage change, version bump, publication, or tag. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 23:17:35 -05:00
parent 4919db6168
commit e083c44e05
22 changed files with 14580 additions and 6 deletions
@@ -0,0 +1,144 @@
+# Rebuilding `@img/sharp-libvips-linux-x64` 1.3.4
+
+This directory is the corresponding source for one redistributed native binary. These instructions
+say how to rebuild it from the source here, and how to use a rebuilt or modified library in place of
+the redistributed one. They are technical instructions. They are not a legal opinion, and they have
+not been legally certified.
+
+## What this corresponds to
+
+| | |
+| --- | --- |
+| Package | `@img/sharp-libvips-linux-x64` 1.3.4, from the npm registry |
+| Declared licence | `LGPL-3.0-or-later` |
+| Content digest | `ekklesia-content-v1:sha256:2927fe11493461fa34847191d0b9b1c6441007c3e99f306e5708cb07fb0a5e59` |
+| Binary | `lib/libvips-cpp.so.8.18.7` (SONAME `libvips-cpp.so.8.18.7`) |
+| Target | Linux, x86-64, glibc 2.28 or newer |
+| Used by | `sharp` 0.35.5, through `@img/sharp-linux-x64` 0.35.5 |
+
+`lib/libvips-cpp.so.8.18.7` is one shared object. libvips and the libraries it uses are built as
+static libraries and linked into it: 27 C and C++ libraries, glib's bundled gvdb, librsvg with its
+Rust crates, and the Rust standard library. `recipe.json` lists every component, its licence, its
+source file and patches with their SHA-256, and its relationship to the binary. `PROVENANCE.md`
+records how all of this was established, and `licenses/` holds the reviewed licence and notice
+material.
+
+## What is here
+
+- `sources/sharp-libvips-1.3.4.tar.gz`: the build definitions that produced the binary. They are
+  `build.sh`, `build/posix.sh`, `platforms/linux-x64/` (a Dockerfile and its toolchain files), and
+  `versions.properties`.
+- `sources/*.tar.*`: the source archive of every compiled library, exactly as `build/posix.sh`
+  downloads it.
+- `patches/`: the five patches `build/posix.sh` applies: glib, mozjpeg, librsvg, libultrahdr, and
+  libvips.
+- `sources/crates/`: all 353 Rust crates librsvg's resolved `Cargo.lock` records.
+  - 147 are compiled into the binary.
+  - 17 are needed only to build it: build-script dependencies, and one procedural macro that adds no
+    code.
+  - 189 serve librsvg's other workspace members, its tests, or other platforms. Cargo resolves the
+    whole workspace lockfile even when it builds one package, so it needs them present to build
+    offline. None of them is compiled into the binary.
+
+  `recipe.json` records each crate's role.
+- `sources/rust/rust-src-nightly-2026-09-27.tar.xz`: the standard-library source of the Rust toolchain
+  that built librsvg, `rustc 1.101.0-nightly (75a75c3e0 2026-09-26)`.
+- `build/`: the resolved librsvg `Cargo.lock`, and the two scripts below that point the unchanged build
+  logic at these files.
+
+Every file's SHA-256 is in `recipe.json` and in the archive's top-level `corresponding-source.json`.
+Check them before building.
+
+## How the original was built
+
+sharp-libvips builds the binary in a Rocky Linux 8 container, which pins glibc 2.28:
+
+```sh
+./build.sh linux-x64
+#   = docker build --pull -t vips-dev-linux-x64 platforms/linux-x64
+#     docker run --rm -v "$PWD:/packaging" vips-dev-linux-x64 sh -c /packaging/build/posix.sh
+```
+
+`platforms/linux-x64/Dockerfile` installs the toolchain: gcc-toolset-15, cmake, nasm, pkgconf, meson
+and ninja (from pip), a nightly Rust toolchain through rustup, and cargo-c. `build/posix.sh` then
+downloads each library's archive, applies the patches, builds every library as a static library into
+`/target`, and links libvips-cpp. It packs the result as `sharp-libvips-linux-x64.tar.gz`, whose
+`lib/libvips-cpp.so.8.18.7` is the file the npm package ships. Meson runs with
+`wrap_mode = 'nofallback'` (`platforms/linux-x64/meson.ini`), so it downloads no subproject. The only
+forced fallback, gvdb, builds from the copy inside glib's archive.
+
+## Rebuilding from this source
+
+You need a Linux x86-64 host with Docker. Building the container image needs network access to fetch
+the toolchain (Rocky Linux packages, rustup, pip, and crates.io for cargo-c). Source for the library
+itself comes only from this directory.
+
+1. **Unpack the build definitions.**
+
+   ```sh
+   tar -xzf sources/sharp-libvips-1.3.4.tar.gz      # creates sharp-libvips-1.3.4/
+   ```
+
+2. **Point the build at this source.**
+
+   ```sh
+   sh build/prepare-offline.sh sharp-libvips-1.3.4
+   ```
+
+   It copies the archives and patches into `sharp-libvips-1.3.4/offline/`. It unpacks the crates into
+   a vendored directory source; Cargo checks each against the lockfile's checksum. It then makes three
+   mechanical edits:
+   - in `build/posix.sh`, `CURL` becomes `offline/offline-curl.sh`. That script answers each URL the
+     script would download with its local copy, using `build/url-map.tsv`, and fails for any other URL;
+   - in `build/posix.sh`, `cargo update --workspace` becomes a copy of the resolved `Cargo.lock`. Cargo
+     is set offline and uses the vendored crates;
+   - in `platforms/linux-x64/Dockerfile`, the Rust toolchain is pinned to `nightly-2026-09-27`, the
+     toolchain whose `rustc` built the original. The upstream file installs whatever nightly is
+     current.
+
+   Nothing that is compiled changes: every patch is still applied by `build/posix.sh`, exactly as
+   upstream.
+
+3. **Build.**
+
+   ```sh
+   cd sharp-libvips-1.3.4
+   ./build.sh linux-x64
+   ```
+
+   The result is `sharp-libvips-1.3.4/sharp-libvips-linux-x64.tar.gz`. Its
+   `lib/libvips-cpp.so.8.18.7` is the rebuilt library. A full build takes from tens of minutes to an
+   hour, depending on the host.
+
+**Modifying a component.** Before step 3, change the source the build uses. Then `build/posix.sh`
+compiles your version in place of the original:
+- for a C or C++ library, replace its archive in `offline/files/` under the same name, or add a patch
+  to the matching step of `build/posix.sh`;
+- for a Rust crate, edit it under `offline/vendor/`. The `.cargo-checksum.json` files list no
+  per-file checksums, so an edited crate still builds.
+
+The result is not byte-identical to the redistributed binary: a compiler toolchain, timestamps, and
+paths that were not recorded can all differ. It is built from the same source, by the same
+definitions, for the same target.
+
+## Using a rebuilt or modified library
+
+sharp's native addon, `@img/sharp-linux-x64/lib/sharp-linux-x64-0.35.5.node`, links
+`libvips-cpp.so.8.18.7` dynamically. Its RPATH begins with `$ORIGIN/../../sharp-libvips-linux-x64/lib`.
+In an installed or extracted Ekklesia Module Package, that resolves to:
+
+```text
+<module directory>/node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.8.18.7
+```
+
+To use your library, replace that file with your build's `lib/libvips-cpp.so.8.18.7`. Keep the file
+name and SONAME, and keep libvips's C++ API compatible with libvips 8.18, which sharp 0.35.5 is
+compiled against. Then restart the process that loads the module. No other file changes, and the
+addon does not need rebuilding.
+
+`ldd <module directory>/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64-0.35.5.node` shows which
+`libvips-cpp.so.8.18.7` the addon resolves.
+
+A module package's own `ekklesia.package.json` records the SHA-256 of each file it shipped, so a
+modified tree is no longer the package Ekklesia verified at installation. Install it as your own
+build.
@@ -0,0 +1,25 @@
+#!/bin/sh
+# Stands in for curl in sharp-libvips's build/posix.sh, so the build reads only this artifact's
+# sources. Given the URL posix.sh would download, it prints the recorded local copy, or with -O
+# writes it to the current directory under the URL's file name. A URL with no recorded copy fails:
+# the build never reaches the network for source.
+set -eu
+write=""
+url=""
+for argument in "$@"; do
+  case "$argument" in
+    -O) write=yes ;;
+    -*) ;;
+    *) url="$argument" ;;
+  esac
+done
+file="$(awk -F '\t' -v url="$url" '$1 == url { print $2 }' /packaging/offline/url-map.tsv)"
+if [ -z "$file" ]; then
+  echo "offline-curl: no recorded source for $url" >&2
+  exit 22
+fi
+if [ -n "$write" ]; then
+  cp "/packaging/offline/files/$file" "./${url##*/}"
+else
+  cat "/packaging/offline/files/$file"
+fi
@@ -0,0 +1,61 @@
+#!/bin/sh
+# Prepares an extracted sharp-libvips v1.3.4 source tree to build lib/libvips-cpp.so.8.18.7 from this
+# artifact's sources alone. BUILDING.md describes each step.
+#
+#   sh build/prepare-offline.sh <extracted sharp-libvips-1.3.4 directory>
+#
+# Only the source acquisition changes; nothing that is compiled does:
+#   - every archive and patch build/posix.sh downloads is read from this artifact (offline-curl.sh);
+#   - librsvg's Rust crates come from this artifact as a vendored directory source, and its lockfile
+#     is the resolved one recorded here instead of a fresh `cargo update` against crates.io;
+#   - platforms/linux-x64/Dockerfile installs the exact nightly Rust toolchain the binary records.
+set -eu
+here="$(cd "$(dirname "$0")/.." && pwd)"
+tree="$(cd "$1" && pwd)"
+[ -f "$tree/build/posix.sh" ] && [ -f "$tree/platforms/linux-x64/Dockerfile" ] || {
+  echo "prepare-offline: $1 is not an extracted sharp-libvips source tree." >&2
+  exit 1
+}
+offline="$tree/offline"
+rm -rf "$offline"
+mkdir -p "$offline/files" "$offline/vendor"
+
+cp "$here"/sources/*.tar.* "$here"/patches/*.patch "$offline/files/"
+cp "$tree/THIRD-PARTY-NOTICES.md" "$offline/files/THIRD-PARTY-NOTICES.md"
+cp "$here/build/url-map.tsv" "$here/build/Cargo.lock" "$offline/"
+cp "$here/build/offline-curl.sh" "$offline/offline-curl.sh"
+chmod 755 "$offline/offline-curl.sh"
+
+for crate in "$here"/sources/crates/*.crate; do
+  tar -xzf "$crate" -C "$offline/vendor"
+  name="$(basename "$crate" .crate)"
+  checksum="$(sha256sum "$crate" | cut -d ' ' -f 1)"
+  printf '{"files":{},"package":"%s"}\n' "$checksum" > "$offline/vendor/$name/.cargo-checksum.json"
+done
+cat > "$offline/cargo-config.toml" <<'CONFIG'
+[source.crates-io]
+replace-with = "vendored-sources"
+
+[source.vendored-sources]
+directory = "/packaging/offline/vendor"
+
+[net]
+offline = true
+CONFIG
+
+posix="$tree/build/posix.sh"
+grep -q '^CURL="curl ' "$posix" && grep -q '^cargo update --workspace$' "$posix" || {
+  echo "prepare-offline: build/posix.sh is not the v1.3.4 script this artifact was prepared for." >&2
+  exit 1
+}
+sed -i \
+  -e 's|^CURL="curl .*"$|CURL="/packaging/offline/offline-curl.sh"\nexport CARGO_NET_OFFLINE=true\nmkdir -p "${CARGO_HOME:-$HOME/.cargo}" \&\& cp /packaging/offline/cargo-config.toml "${CARGO_HOME:-$HOME/.cargo}/config.toml"|' \
+  -e 's|^cargo update --workspace$|cp /packaging/offline/Cargo.lock Cargo.lock|' \
+  "$posix"
+dockerfile="$tree/platforms/linux-x64/Dockerfile"
+grep -q -- '--default-toolchain nightly ' "$dockerfile" || {
+  echo "prepare-offline: platforms/linux-x64/Dockerfile is not the v1.3.4 one." >&2
+  exit 1
+}
+sed -i 's|--default-toolchain nightly |--default-toolchain nightly-2026-09-27 |' "$dockerfile"
+echo "prepare-offline: $tree builds from this artifact's sources."
@@ -0,0 +1,33 @@
+https://cairographics.org/releases/cairo-1.18.6.tar.xz	cairo-1.18.6.tar.xz
+https://cairographics.org/releases/pixman-0.46.4.tar.gz	pixman-0.46.4.tar.gz
+https://download.gnome.org/sources/glib/2.90/glib-2.90.0.tar.xz	glib-2.90.0.tar.xz
+https://download.gnome.org/sources/librsvg/2.63/librsvg-2.63.2.tar.xz	librsvg-2.63.2.tar.xz
+https://download.gnome.org/sources/libxml2/2.15/libxml2-2.15.4.tar.xz	libxml2-2.15.4.tar.xz
+https://download.gnome.org/sources/pango/1.58/pango-1.58.2.tar.xz	pango-1.58.2.tar.xz
+https://gist.github.com/kleisauke/284d685efa00908da99ea6afbaaf39ae/raw/bdad5489a61c217850631571caf57f5db6ea8b2c/glib-without-gregex.patch	glib-without-gregex.patch
+https://gist.githubusercontent.com/lovell/313a6901e9db1bf285f2a1f1180499e4/raw/3988223c7dfa4d22745d9392034b0117abef1446/libvips-cpp-soversion.patch	libvips-cpp-soversion.patch
+https://github.com/GNOME/librsvg/commit/9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch	librsvg-9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch
+https://github.com/dloebl/cgif/archive/v0.5.4.tar.gz	cgif-0.5.4.tar.gz
+https://github.com/freetype/freetype/archive/VER-2-14-3.tar.gz	freetype-2.14.3.tar.gz
+https://github.com/fribidi/fribidi/releases/download/v1.0.17/fribidi-1.0.17.tar.xz	fribidi-1.0.17.tar.xz
+https://github.com/google/highway/archive/1.4.0.tar.gz	highway-1.4.0.tar.gz
+https://github.com/google/libultrahdr/archive/v2.0.2.tar.gz	libultrahdr-2.0.2.tar.gz
+https://github.com/harfbuzz/harfbuzz/archive/14.5.0.tar.gz	harfbuzz-14.5.0.tar.gz
+https://github.com/libarchive/libarchive/releases/download/v3.8.9/libarchive-3.8.9.tar.xz	libarchive-3.8.9.tar.xz
+https://github.com/libexif/libexif/releases/download/v0.6.26/libexif-0.6.26.tar.xz	libexif-0.6.26.tar.xz
+https://github.com/libexpat/libexpat/releases/download/R_2_8_5/expat-2.8.5.tar.xz	expat-2.8.5.tar.xz
+https://github.com/libffi/libffi/releases/download/v3.8.0/libffi-3.8.0.tar.gz	libffi-3.8.0.tar.gz
+https://github.com/libvips/libvips/releases/download/v8.18.7/vips-8.18.7.tar.xz	vips-8.18.7.tar.xz
+https://github.com/lovell/libimagequant/archive/v2.4.1.tar.gz	libimagequant-2.4.1.tar.gz
+https://github.com/mm2/Little-CMS/releases/download/lcms2.19.1/lcms2-2.19.1.tar.gz	lcms2-2.19.1.tar.gz
+https://github.com/mozilla/mozjpeg/archive/0826579.tar.gz	mozjpeg-0826579.tar.gz
+https://github.com/mozilla/mozjpeg/commit/f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch	mozjpeg-f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch
+https://github.com/pnggroup/libpng/archive/v1.6.58.tar.gz	libpng-1.6.58.tar.gz
+https://github.com/strukturag/libheif/releases/download/v1.23.5/libheif-1.23.5.tar.gz	libheif-1.23.5.tar.gz
+https://github.com/zlib-ng/zlib-ng/archive/2.3.3.tar.gz	zlib-ng-2.3.3.tar.gz
+https://gitlab.com/libtiff/libtiff/-/archive/v4.7.2/libtiff-v4.7.2.tar.gz	libtiff-v4.7.2.tar.gz
+https://gitlab.freedesktop.org/fontconfig/fontconfig/-/archive/2.18.3/fontconfig-2.18.3.tar.gz	fontconfig-2.18.3.tar.gz
+https://patch-diff.githubusercontent.com/raw/google/libultrahdr/pull/383.patch	libultrahdr-pull-383.patch
+https://raw.githubusercontent.com/lovell/sharp-libvips/main/THIRD-PARTY-NOTICES.md	THIRD-PARTY-NOTICES.md
+https://storage.googleapis.com/aom-releases/libaom-3.15.1.tar.gz	libaom-3.15.1.tar.gz
+https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.6.0.tar.gz	libwebp-1.6.0.tar.gz