21 Commits
Author SHA1 Message Date
ekklesiaandClaude Opus 5.5 9331a8f7df fix(display): retry a display that could not start, and re-resolve a running one periodically
CI / verify (pull_request) Successful in 1m7s
CI / infrastructure-smoke (pull_request) Successful in 38s
CI / package-release (pull_request) Successful in 50s
CI / migration-smoke (pull_request) Successful in 2m9s
CI / image-build (pull_request) Successful in 57s
CI / deploy-smoke (same-origin) (pull_request) Successful in 53s
CI / deploy-smoke (split) (pull_request) Successful in 2m35s
CI / package-publication (pull_request) Successful in 1m1s
A kiosk that booted before its server, as after a power cut, stopped on 'This
display could not be reached' until someone pressed Retry: with the network
up, no online event ever came. It now retries on its own, from two seconds
doubling to a minute. A running display also re-resolves its descriptor every
five minutes in the background, so a module upgraded on the server is picked
up without a reload, and the same answer disturbs nothing.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 21:45:37 -05:00
ekklesiaandClaude Opus 5.5 8f90871be0 docs(deploy): document how to set up an unattended display kiosk
Stage 0 requires that kiosk setup can be reproduced from documentation, and
there was none. Cover the address, a single-launcher Chromium command with a
persistent profile, a systemd user unit, pairing and the fixed 180-day device
credential, what the screen does through each kind of outage, the known offline
cold-start limit, and why a second launcher produces duplicate windows.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 21:45:37 -05:00
ekklesiaandClaude Opus 5.5 85b1377fd3 fix(devices): confirm before revoking a display
Revoke acted on a single click, yet it strands the screen until someone
re-pairs it in person. Explain the consequence in the shared in-product
Dialog first (DESIGN 21.4), with an obvious Cancel.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 21:31:36 -05:00
ekklesiaandClaude Opus 5.5 4bcdb578e6 fix(display): keep a live display across reconnects, and return a revoked one to pairing
Every browser 'online' event reset the display host to its loading state and
mounted the module again: a fresh viewer session, a fresh stage, and every
cached object URL released, on each Wi-Fi blip. And a device revoked while its
display was open never noticed; it kept playing cached content and polling
403s until someone reloaded the page.

The host now re-resolves its descriptor in the background (ADR 0043) and only
replaces the frame when the answer actually changed: a new version or
principal remounts, a refusal returns the screen to pairing (ADR 0036), and
any other failure keeps the running display. A module request refused with
401/403 prompts the same revalidation, throttled, so revocation is noticed
within one heartbeat. The offline indicator waits ten seconds before
appearing, so a momentary blip shows nothing, and no longer names the
platform on a lobby wall.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 21:31:36 -05:00
ekklesiaandClaude Opus 5.5 06bad2fe36 docs: bring CLAUDE.md in line with the current repositories and product rules
Core's agent guidance still described modules living in this monorepo and
manifest v7, and said nothing about the product rules that now govern every
change. Record that first-party modules live in their own Zayin repositories
and install as Module Packages, that the parent ROADMAP.md and DESIGN.md are
authoritative for product direction, the ownership, branding, token, dark-mode
and no-native-dialog rules, that display pairing is transitional until Stage 1,
and the branch-per-PR, no-push-to-main, and no-release-tag workflow.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-12 21:31:36 -05:00
The_River ae976ca229 Merge pull request 'chore(release): prepare 0.2.4' (#91) from chore/release-0.2.4 into main
CI / verify (push) Successful in 1m13s
CI / infrastructure-smoke (push) Successful in 38s
CI / package-release (push) Successful in 49s
CI / image-build (push) Successful in 30s
CI / migration-smoke (push) Successful in 2m13s
CI / deploy-smoke (same-origin) (push) Successful in 47s
CI / package-publication (push) Successful in 1m4s
CI / deploy-smoke (split) (push) Successful in 2m23s
Reviewed-on: https://git.riverofglorymn.org/River_of_Glory_Church/Ekklesia/pulls/91
Reviewed-by: The_River <[email protected]>
2026-10-10 17:09:10 +00:00
ekklesia f9585cc7ec chore(release): prepare 0.2.4
CI / verify (pull_request) Successful in 1m5s
CI / infrastructure-smoke (pull_request) Successful in 36s
CI / migration-smoke (pull_request) Successful in 1m51s
CI / package-release (pull_request) Successful in 34s
CI / image-build (pull_request) Successful in 59s
CI / deploy-smoke (same-origin) (pull_request) Successful in 47s
CI / deploy-smoke (split) (pull_request) Successful in 2m21s
CI / package-publication (pull_request) Successful in 49s
2026-10-10 11:54:00 -05:00
The_River 6dae29412f Merge pull request 'test(deploy): exercise external module packages' (#90) from test/external-module-deploy-smoke into main
CI / verify (push) Successful in 1m3s
CI / infrastructure-smoke (push) Successful in 35s
CI / migration-smoke (push) Successful in 1m54s
CI / package-release (push) Successful in 34s
CI / image-build (push) Successful in 26s
CI / deploy-smoke (same-origin) (push) Successful in 40s
CI / deploy-smoke (split) (push) Successful in 2m13s
CI / package-publication (push) Successful in 51s
Reviewed-on: https://git.riverofglorymn.org/River_of_Glory_Church/Ekklesia/pulls/90
Reviewed-by: The_River <[email protected]>
2026-10-10 16:32:32 +00:00
ekklesia 40ef0ff802 fix(deploy): bound object storage allocation size
CI / verify (pull_request) Successful in 1m8s
CI / infrastructure-smoke (pull_request) Successful in 35s
CI / migration-smoke (pull_request) Successful in 1m49s
CI / package-release (pull_request) Successful in 35s
CI / image-build (pull_request) Successful in 50s
CI / deploy-smoke (same-origin) (pull_request) Successful in 43s
CI / deploy-smoke (split) (pull_request) Successful in 2m19s
CI / package-publication (pull_request) Successful in 52s
2026-10-10 11:07:53 -05:00
ekklesia a0e9948820 fix(storage): recognize errors across module packages
CI / verify (pull_request) Successful in 1m7s
CI / infrastructure-smoke (pull_request) Successful in 29s
CI / migration-smoke (pull_request) Successful in 1m44s
CI / package-release (pull_request) Successful in 35s
CI / image-build (pull_request) Successful in 53s
CI / deploy-smoke (same-origin) (pull_request) Successful in 44s
CI / deploy-smoke (split) (pull_request) Successful in 2m20s
CI / package-publication (pull_request) Successful in 51s
2026-10-10 10:54:13 -05:00
ekklesia 6e64b3c5f9 test(deploy): allow slower display uploads
CI / verify (pull_request) Successful in 1m8s
CI / infrastructure-smoke (pull_request) Successful in 35s
CI / migration-smoke (pull_request) Successful in 1m57s
CI / package-release (pull_request) Successful in 37s
CI / image-build (pull_request) Successful in 54s
CI / deploy-smoke (same-origin) (pull_request) Successful in 50s
CI / deploy-smoke (split) (pull_request) Successful in 2m21s
CI / package-publication (pull_request) Successful in 52s
2026-10-09 22:37:18 -05:00
ekklesia fa27603133 test(deploy): exercise external module packages
CI / verify (pull_request) Successful in 1m13s
CI / infrastructure-smoke (pull_request) Successful in 40s
CI / migration-smoke (pull_request) Successful in 1m56s
CI / package-release (pull_request) Successful in 37s
CI / image-build (pull_request) Successful in 54s
CI / deploy-smoke (same-origin) (pull_request) Successful in 50s
CI / deploy-smoke (split) (pull_request) Successful in 2m22s
CI / package-publication (pull_request) Successful in 55s
2026-10-09 22:18:04 -05:00
The_River 9e01679fcb Merge pull request 'chore(release): prepare 0.2.3' (#89) from chore/release-0.2.3 into main
CI / verify (push) Successful in 1m8s
CI / infrastructure-smoke (push) Successful in 28s
CI / migration-smoke (push) Successful in 1m56s
CI / package-release (push) Successful in 37s
CI / image-build (push) Successful in 25s
CI / deploy-smoke (same-origin) (push) Successful in 49s
CI / deploy-smoke (split) (push) Successful in 1m22s
CI / package-publication (push) Successful in 50s
Reviewed-on: https://git.riverofglorymn.org/River_of_Glory_Church/Ekklesia/pulls/89
Reviewed-by: The_River <[email protected]>
2026-10-10 00:01:54 +00:00
ekklesia 227c92e35d chore(release): prepare 0.2.3
CI / verify (pull_request) Successful in 1m5s
CI / infrastructure-smoke (pull_request) Successful in 36s
CI / migration-smoke (pull_request) Successful in 1m54s
CI / package-release (pull_request) Successful in 35s
CI / image-build (pull_request) Successful in 1m1s
CI / deploy-smoke (same-origin) (pull_request) Successful in 45s
CI / deploy-smoke (split) (pull_request) Successful in 1m30s
CI / package-publication (pull_request) Successful in 55s
2026-10-09 18:38:22 -05:00
The_River 505c9e8c0a Merge pull request 'Fix native package verification on hardened noexec /tmp' (#88) from fix/verifier-exec-scratch into main
CI / verify (push) Successful in 1m9s
CI / infrastructure-smoke (push) Successful in 37s
CI / migration-smoke (push) Successful in 2m1s
CI / package-release (push) Successful in 36s
CI / image-build (push) Successful in 26s
CI / deploy-smoke (same-origin) (push) Successful in 44s
CI / deploy-smoke (split) (push) Successful in 1m27s
CI / package-publication (push) Successful in 54s
Reviewed-on: https://git.riverofglorymn.org/River_of_Glory_Church/Ekklesia/pulls/88
Reviewed-by: The_River <[email protected]>
2026-10-09 23:36:11 +00:00
ekklesia 4be046c8d0 fix: support native package verifier scratch
CI / verify (pull_request) Successful in 1m10s
CI / infrastructure-smoke (pull_request) Successful in 37s
CI / migration-smoke (pull_request) Successful in 1m54s
CI / package-release (pull_request) Successful in 36s
CI / image-build (pull_request) Successful in 52s
CI / deploy-smoke (same-origin) (pull_request) Successful in 51s
CI / deploy-smoke (split) (pull_request) Successful in 1m31s
CI / package-publication (pull_request) Successful in 51s
2026-10-09 18:05:04 -05:00
The_River c5a6cdaf6a Merge pull request 'chore(release): prepare 0.2.2' (#87) from chore/release-0.2.2 into main
CI / verify (push) Successful in 1m7s
CI / infrastructure-smoke (push) Successful in 37s
CI / migration-smoke (push) Successful in 1m54s
CI / package-release (push) Successful in 35s
CI / image-build (push) Successful in 12s
CI / deploy-smoke (same-origin) (push) Successful in 49s
CI / deploy-smoke (split) (push) Successful in 1m22s
CI / package-publication (push) Successful in 1m0s
Reviewed-on: https://git.riverofglorymn.org/River_of_Glory_Church/Ekklesia/pulls/87
Reviewed-by: The_River <[email protected]>
2026-10-09 22:07:46 +00:00
ekklesia c4fded6ecf chore(release): prepare 0.2.2
CI / verify (pull_request) Successful in 1m6s
CI / infrastructure-smoke (pull_request) Successful in 35s
CI / migration-smoke (pull_request) Successful in 1m53s
CI / package-release (pull_request) Successful in 35s
CI / image-build (pull_request) Successful in 31s
CI / deploy-smoke (same-origin) (pull_request) Successful in 45s
CI / deploy-smoke (split) (pull_request) Successful in 1m31s
CI / package-publication (pull_request) Successful in 53s
2026-10-09 16:55:18 -05:00
The_River 9ad2d5b564 Merge pull request 'feat: corresponding-source artifacts for native packages that redistribute copyleft code (ADR 0055)' (#86) from feat/corresponding-source into main
CI / verify (push) Successful in 1m6s
CI / infrastructure-smoke (push) Successful in 37s
CI / migration-smoke (push) Successful in 1m58s
CI / package-release (push) Successful in 38s
CI / image-build (push) Successful in 11s
CI / deploy-smoke (same-origin) (push) Successful in 50s
CI / deploy-smoke (split) (push) Successful in 1m22s
CI / package-publication (push) Successful in 51s
Reviewed-on: https://git.riverofglorymn.org/River_of_Glory_Church/Ekklesia/pulls/86
Reviewed-by: The_River <[email protected]>
2026-10-09 04:34:50 +00:00
ekklesiaandClaude Opus 5.5 98d93cc12d docs: accept ADR 0052, whose supplement mechanism is fully implemented
CI / verify (pull_request) Successful in 1m10s
CI / infrastructure-smoke (pull_request) Successful in 30s
CI / migration-smoke (pull_request) Successful in 1m53s
CI / package-release (pull_request) Successful in 1m39s
CI / image-build (pull_request) Successful in 23s
CI / deploy-smoke (same-origin) (pull_request) Successful in 49s
CI / deploy-smoke (split) (pull_request) Successful in 1m29s
CI / package-publication (pull_request) Successful in 50s
Every decision in ADR 0052 is implemented and tested in @ekklesia/module-pack:
- package material first, and a supplement only adds to it;
- reviewed, generic supplement data, checked in with the packager;
- an exact name, version, and content-digest match, failing closed on any mismatch;
- the documented, host-independent digest;
- a deterministic, labelled merge;
- no licence override, with a consistency check;
- no network during packaging;
- packageFormat 1.

The libvips 1.3.3 and 1.3.4 supplements are curated and shipped. The source-obligation determination
it deferred is now decided by ADR 0055. Only the status line and the index entry change; the decision
text is untouched.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 23:24:16 -05:00
ekklesiaandClaude Opus 5.5 e083c44e05 feat: corresponding-source artifacts for native packages that redistribute copyleft code (ADR 0055)
CI / verify (pull_request) Successful in 1m3s
CI / infrastructure-smoke (pull_request) Successful in 36s
CI / migration-smoke (pull_request) Successful in 1m53s
CI / package-release (pull_request) Successful in 36s
CI / package-publication (pull_request) Canceled after 0s
CI / deploy-smoke (same-origin) (pull_request) Canceled after 0s
CI / deploy-smoke (split) (pull_request) Canceled after 0s
CI / image-build (pull_request) Canceled after 30s
Ekklesia may publicly distribute a native Module Package containing LGPL or MPL components only beside
a version-specific corresponding-source artifact. The artifact carries the exact source, patches,
build definitions, and rebuild and relink instructions for those components. ADR 0055 (Accepted)
records the policy: project compliance policy, not legal certification. It decides the item ADR 0052
deferred, and extends ADR 0049's distributable-release gate. Module Package v1, packageFormat 1, and
both libvips supplements are unchanged.

The mechanism is generic and lives in @ekklesia/module-pack:
- Source recipes: reviewed data under corresponding-sources/<package>/<version>/. recipe.json names
  the exact package (name, version, content digest) and the exact supplement it was prepared with,
  and lists every component with its role, licence, licence material, and relationship to the
  binary. Every source file and patch is recorded by origin, size, and SHA-256; documents are hashed
  too, and every exclusion has a reason. Loading is strict and refuses anything unlisted.
- Acquisition (the only network step) fetches each recorded file once into a cache keyed by
  SHA-256. It uses node:https with exact headers and no decoding, retries transient failures, and
  stores only bytes that match.
- Construction is offline. From a built package it finds the covered native dependencies (by
  declared licence, or by a supplement component's licence), verifies every byte, and writes four
  sibling artifacts: the deterministic <package>-corresponding-source.tar.gz and its .sha256;
  <package>.redistribution.json, linking package, dependencies, supplement, recipe, and source by
  digest; and <package>.source-notice.txt, the release notice.
- The public-distribution gate (verifyRedistribution, `ekklesia-corresponding-source verify`)
  approves only when the recipe, supplement, canonical archive, manifest, and metadata identify
  exactly these files. It refuses bundled copyleft code, and approves packages with no covered
  native code without an artifact. packModule is untouched.

First complete proof: @img/[email protected] (sharp 0.35.5), from the reviewed supplement
and provenance.
- 383 components and 387 source files, about 208 MB: the sharp-libvips v1.3.4 build definitions,
  the 27 library archives, the five build-time patches, the 353 crates librsvg's resolved Cargo.lock
  records, and rust-src for nightly-2026-09-27 (rustc 75a75c3e0), which vendors std's dependencies.
- BUILDING.md and two helpers point sharp-libvips's unchanged build logic at those files.
- Followed in a Rocky Linux 8 container with networking disabled, they rebuilt
  libvips-cpp.so.8.18.7, byte-identically across two builds. Swapped into a verified, extracted
  native package, the rebuilt library was the one loaded, and sharp ran.
- The native proof's source artifact, native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz,
  is 207242785 bytes, sha256 cc405492032b04a5391ab84e64f0e5e50f78e571ef7818204782c25a03eb50b7. It is
  byte-identical from unrelated directories and from a cache acquired fresh from upstream.

Tests:
- corresponding-source.test.ts runs in CI: synthetic inputs, every negative case, acquisition with
  retries and refusal, and determinism.
- corresponding-source-native.test.ts checks the real recipe everywhere. With a verified cache
  (`pnpm test:corresponding-source`, which acquires once) it builds the real artifact twice, gates
  it, fails it closed, and runs sharp from the package.

Docs: the module guide's distribution section, the README and architecture notes, and an ADR 0052
cross-reference.

No Signage change, version bump, publication, or tag.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 23:17:35 -05:00
59 changed files with 16120 additions and 171 deletions
+57 -15
View File
@@ -6,27 +6,60 @@ contents, because restated details drift.
## What this repository is ## What this repository is
Ekklesia is a modular, self-hosted church platform built as a strictly layered pnpm/TypeScript Ekklesia is a modular, self-hosted church platform. **This repository is the platform** — Core, the
monorepo: API, the Dashboard and display shell, and the public `@ekklesia/*` module-development packages — built
as a strictly layered pnpm/TypeScript monorepo:
- **Core** (`core/`) owns platform infrastructure: identity, database connectivity, migration - **Core** (`core/`) owns platform infrastructure: identity, database connectivity, migration
execution, module discovery/lifecycle, and other feature-agnostic services. Core never imports a execution, module discovery/lifecycle, devices, and other feature-agnostic services. Core never
feature module, a surface, or an app. imports a feature module, a surface, or an app.
- **Modules** (`modules/*`, template at `templates/module`) are independently versioned product - **Modules** are independently versioned product packages. First-party modules live in **their own
packages. They own their own data and migrations, declare capabilities/permissions through repositories** (Signage is `Zayin/Signage`), build against the published `@ekklesia/*` packages
`@ekklesia/contracts` and `@ekklesia/module-sdk`, and must never import Core, another module, or a (ADR 0053), and reach an installation as a Module Package (ADR 0049) installed into module storage
surface directly. Per ADR 0021 a module owns its UI; manifest v7 permits independently optional outside the image (ADR 0048). No module source lives in this repository; `templates/module` is the
primary and full-viewport display artifacts, authored in React against the same role-neutral example, and `modules/` is only an optional, ignored local installed-modules directory. A module owns
`@ekklesia/module-ui-sdk` ABI. Ekklesia mounts each through a separate surface role, while its data, migrations, and **its own domain UI** (ADR 0021), declares capabilities/permissions through
`pnpm boundaries` still rejects every non-React UI framework and non-`.tsx` presentation source. `@ekklesia/contracts` and `@ekklesia/module-sdk`, and never imports Core, another module, or a
surface. Manifest v8 permits independently optional primary and full-viewport display artifacts,
authored in React against the role-neutral `@ekklesia/module-ui-sdk` ABI; database access is the
Module Database ABI v1 (ADR 0050).
- **Surfaces** (`surfaces/*`) own the containing application shell, navigation, and every - **Surfaces** (`surfaces/*`) own the containing application shell, navigation, and every
presentation concern a module does not. They consume public API/client contracts only, never Core presentation concern a module does not. The shell **hosts** module UI through the generic loader;
or module internals directly. it never rebuilds a module's domain screens. Surfaces consume public API/client contracts only.
These boundaries are enforced by `pnpm boundaries` (`scripts/check-boundaries.ts`), not just These boundaries are enforced by `pnpm boundaries` (`scripts/check-boundaries.ts`), not just
documented. Read `docs/architecture/repository-boundaries.md` and documented. Read `docs/architecture/repository-boundaries.md` and
`docs/architecture/platform-architecture.md` before proposing any change that crosses a layer. `docs/architecture/platform-architecture.md` before proposing any change that crosses a layer.
## Product direction lives beside this repository
The current roadmap and product/interface design contract are `ROADMAP.md` and `DESIGN.md` in the
parent development directory (`../ROADMAP.md` and `../DESIGN.md` from a checkout in the standard
`~/development/ekklesia/` layout). They are deliberately **not** committed here; do not copy them into
this repository to make them available. They are authoritative for product direction: read the
relevant stages and sections before designing a change. If this repository's documentation disagrees
with them, flag the conflict, correct the repository-local documentation where appropriate, and never
silently follow the stale text. Architectural decisions still change only through ADRs (below).
Rules from those documents that every change here must respect:
- **Ownership.** Modules own their domain data, behaviour, and UI. Shared resources — displays and
devices, accounts and identity, roles and authorization, branding, mail, push, storage providers,
and module installation/lifecycle — belong to the platform, and modules consume them.
- **Branding.** Normal user-facing UI never hard-codes "Ekklesia"; product identity comes from
installation branding (ADR 0046). Package names, diagnostics, logs, and developer tooling may.
Existing hard-coded shell strings are known debt for Stage 1, not a pattern to copy.
- **Visual direction.** Simple, minimal, and calm; responsive for phone, tablet, and desktop; light,
dark, and system modes. Normal UI colours come only from the shared design tokens
(`@ekklesia/design-tokens`, ADR 0039) — never a hard-coded colour.
- **No browser-native dialogs.** `window.alert`, `window.confirm`, and `window.prompt` are prohibited
in product UI. Destructive actions use in-product confirmation proportional to their impact.
- **Displays are transitional.** Displays are platform-owned. Today's device pairing (the Dashboard
sidebar's *Display devices* panel) and Signage's own display roster are a transitional
implementation that Stage 1 migrates to **Settings → Displays**. Fix defects in them, but do not
build the Stage 1 Settings, global Displays, theme editor, module installer, or responsive shell
redesign early, and keep existing paired displays working across any change.
## ADRs are authoritative ## ADRs are authoritative
`docs/architecture/decisions/` records durable architectural decisions. **Accepted ADRs are `docs/architecture/decisions/` records durable architectural decisions. **Accepted ADRs are
@@ -43,6 +76,7 @@ At minimum, read the current versions of:
- `docs/architecture/decisions/README.md` and every ADR relevant to your change - `docs/architecture/decisions/README.md` and every ADR relevant to your change
- `docs/architecture/platform-architecture.md` and `docs/architecture/repository-boundaries.md` - `docs/architecture/platform-architecture.md` and `docs/architecture/repository-boundaries.md`
- `README.md` (current platform status — what is and is not implemented) - `README.md` (current platform status — what is and is not implemented)
- `../ROADMAP.md` and `../DESIGN.md` (current product direction; see above)
- `docs/development/repository-workflow.md` and `docs/development/testing.md` - `docs/development/repository-workflow.md` and `docs/development/testing.md`
- `CONTRIBUTING.md` - `CONTRIBUTING.md`
@@ -52,8 +86,16 @@ your own memory of a prior session** — treat stale assumptions as a bug to cat
## Working conventions ## Working conventions
- Work on a clean branch off an up-to-date `main`; never commit or push directly to `main`; never - The canonical repositories are on the church Gitea under the `Zayin` organization:
change branch-protection settings. `ssh://[email protected]:2222/Zayin/Ekklesia.git` (this one), `Zayin/Signage.git`, and
`Zayin/Classroom_Desktop.git`. A local clone may be behind the server: `git fetch --prune --tags`
and branch from `origin/main`, never from whatever is checked out.
- One branch per PR, off an up-to-date `origin/main`. Never commit or push directly to `main`, never
merge your own PR, and never change branch-protection settings.
- Never create or push a release tag. Release tags belong to the human maintainer.
- A change that needs both a platform contract change and a module change is two PRs: Core first, on
its own branch, with the compatibility boundary stated explicitly; then the module. Do not copy
platform logic into a module to avoid making the correct Core change.
- Keep PRs focused on one bounded slice. Record major architectural changes in an ADR rather than - Keep PRs focused on one bounded slice. Record major architectural changes in an ADR rather than
burying them in an implementation. burying them in an implementation.
- Add or update tests when you change a contract or architectural rule. New behavior needs new - Add or update tests when you change a contract or architectural rule. New behavior needs new
+4 -1
View File
@@ -59,7 +59,10 @@ supplied `LICENSE` bytes; dependency material remains separate in `THIRD_PARTY_N
packages may bundle a module-owned Drizzle version without coupling it to Core; packages may bundle a module-owned Drizzle version without coupling it to Core;
legacy-v8 packages retain their transitional exact-version check. Where a redistributed dependency legacy-v8 packages retain their transitional exact-version check. Where a redistributed dependency
omits licence material, the packager may add reviewed licence supplements omits licence material, the packager may add reviewed licence supplements
([ADR 0052](docs/architecture/decisions/0052-reviewed-license-supplements.md)). Purge, other ([ADR 0052](docs/architecture/decisions/0052-reviewed-license-supplements.md)). A native package that
redistributes LGPL or MPL code is distributed publicly only with its corresponding-source artifact,
which `ekklesia-corresponding-source` builds and gates
([ADR 0055](docs/architecture/decisions/0055-corresponding-source-artifacts.md)). Purge, other
operating-system or CPU targets, signing, and any installer or marketplace do not exist yet. operating-system or CPU targets, signing, and any installer or marketplace do not exist yet.
A module can be built in its own repository from Ekklesia's public module-development packages A module can be built in its own repository from Ekklesia's public module-development packages
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/api", "name": "@ekklesia/api",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"private": true, "private": true,
"type": "module", "type": "module",
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/core", "name": "@ekklesia/core",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"private": true, "private": true,
"type": "module", "type": "module",
+13
View File
@@ -25,6 +25,7 @@ class VerifyArgumentsError extends Error {}
const USAGE = const USAGE =
"Usage: verify-module-package <archive> [--sha256 <hex> | --sha256-file <file>] [--extract-to <new directory>]"; "Usage: verify-module-package <archive> [--sha256 <hex> | --sha256-file <file>] [--extract-to <new directory>]";
export const MODULE_VERIFIER_SCRATCH_ENV = "EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR";
export interface VerifyModulePackageArguments { export interface VerifyModulePackageArguments {
readonly archivePath: string; readonly archivePath: string;
@@ -90,6 +91,16 @@ export function parseSha256File(contents: string): string {
return match[1].toLowerCase(); return match[1].toLowerCase();
} }
/** The CLI's operator-facing bridge to Core's otherwise explicit verifier option. */
export function moduleVerifierScratchDirectory(environment: NodeJS.ProcessEnv): string | undefined {
const configured = environment[MODULE_VERIFIER_SCRATCH_ENV];
if (configured === undefined) return undefined;
if (configured === "" || !path.isAbsolute(configured)) {
throw new VerifyArgumentsError(`${MODULE_VERIFIER_SCRATCH_ENV} must be an absolute path.`);
}
return configured;
}
async function readJson(file: string): Promise<Record<string, unknown>> { async function readJson(file: string): Promise<Record<string, unknown>> {
return JSON.parse(await readFile(file, "utf8")) as Record<string, unknown>; return JSON.parse(await readFile(file, "utf8")) as Record<string, unknown>;
} }
@@ -121,6 +132,7 @@ export async function readModulePackagePlatform(
async function main(): Promise<void> { async function main(): Promise<void> {
const parsed = parseVerifyModulePackageArguments(process.argv.slice(2)); const parsed = parseVerifyModulePackageArguments(process.argv.slice(2));
const executionScratchDirectory = moduleVerifierScratchDirectory(process.env);
let expectedSha256 = parsed.expectedSha256; let expectedSha256 = parsed.expectedSha256;
if (parsed.sha256File !== undefined) { if (parsed.sha256File !== undefined) {
let contents: string; let contents: string;
@@ -137,6 +149,7 @@ async function main(): Promise<void> {
const result = await verifyModulePackage({ const result = await verifyModulePackage({
archivePath: parsed.archivePath, archivePath: parsed.archivePath,
platform, platform,
...(executionScratchDirectory === undefined ? {} : { executionScratchDirectory }),
...(expectedSha256 === undefined ? {} : { expectedSha256 }), ...(expectedSha256 === undefined ? {} : { expectedSha256 }),
...(parsed.extractTo === undefined ? {} : { extractTo: parsed.extractTo }), ...(parsed.extractTo === undefined ? {} : { extractTo: parsed.extractTo }),
}); });
@@ -1,6 +1,6 @@
import { fork } from "node:child_process"; import { fork } from "node:child_process";
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { chmod, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { chmod, mkdir, mkdtemp, readFile, realpath, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import path from "node:path"; import path from "node:path";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
@@ -73,7 +73,8 @@ export type ModulePackageVerificationErrorCode =
| "MODULE_PACKAGE_RUNTIME_PACKAGE_INVALID" | "MODULE_PACKAGE_RUNTIME_PACKAGE_INVALID"
| "MODULE_PACKAGE_SDK_INCOMPATIBLE" | "MODULE_PACKAGE_SDK_INCOMPATIBLE"
| "MODULE_PACKAGE_TARGET_INCOMPATIBLE" | "MODULE_PACKAGE_TARGET_INCOMPATIBLE"
| "MODULE_PACKAGE_TRANSITIONAL_UNSUPPORTED"; | "MODULE_PACKAGE_TRANSITIONAL_UNSUPPORTED"
| "MODULE_PACKAGE_VERIFIER_SCRATCH_UNAVAILABLE";
const messages: Readonly<Record<ModulePackageVerificationErrorCode, string>> = Object.freeze({ const messages: Readonly<Record<ModulePackageVerificationErrorCode, string>> = Object.freeze({
MODULE_PACKAGE_DATABASE_RUNTIME_INCOMPATIBLE: MODULE_PACKAGE_DATABASE_RUNTIME_INCOMPATIBLE:
@@ -118,6 +119,8 @@ const messages: Readonly<Record<ModulePackageVerificationErrorCode, string>> = O
MODULE_PACKAGE_TOO_LARGE: "The archive exceeds the module package size limits.", MODULE_PACKAGE_TOO_LARGE: "The archive exceeds the module package size limits.",
MODULE_PACKAGE_TRANSITIONAL_UNSUPPORTED: MODULE_PACKAGE_TRANSITIONAL_UNSUPPORTED:
"The package metadata contains transitional facts this platform does not understand.", "The package metadata contains transitional facts this platform does not understand.",
MODULE_PACKAGE_VERIFIER_SCRATCH_UNAVAILABLE:
"The verifier execution scratch directory is missing, not writable, or mounted noexec.",
MODULE_PACKAGE_UNREADABLE: "The file is not a readable gzip-compressed ustar archive.", MODULE_PACKAGE_UNREADABLE: "The file is not a readable gzip-compressed ustar archive.",
MODULE_PACKAGE_UNSAFE_MODE: "The archive contains a setuid, setgid, or sticky entry.", MODULE_PACKAGE_UNSAFE_MODE: "The archive contains a setuid, setgid, or sticky entry.",
MODULE_PACKAGE_UNSAFE_PATH: "The archive contains an absolute, escaping, or non-portable path.", MODULE_PACKAGE_UNSAFE_PATH: "The archive contains an absolute, escaping, or non-portable path.",
@@ -157,6 +160,12 @@ export interface VerifyModulePackageOptions {
readonly platform: ModulePackagePlatform; readonly platform: ModulePackagePlatform;
/** A directory that must not exist yet; the verified module directory is written there. */ /** A directory that must not exist yet; the verified module directory is written there. */
readonly extractTo?: string; readonly extractTo?: string;
/**
* Existing writable directory where backend candidates may execute. Defaults to `os.tmpdir()`.
* A hardened caller whose system temporary directory is `noexec` must provide a dedicated
* executable scratch mount; each verification creates and removes its own child directory.
*/
readonly executionScratchDirectory?: string;
readonly loadTimeoutMs?: number; readonly loadTimeoutMs?: number;
} }
@@ -414,12 +423,66 @@ function probeEntrypointPath(): string {
return path.join(path.dirname(self), `module-package-probe${path.extname(self)}`); return path.join(path.dirname(self), `module-package-probe${path.extname(self)}`);
} }
function decodedMountPath(value: string): string {
return value.replace(/\\([0-7]{3})/gu, (_match, octal: string) =>
String.fromCharCode(Number.parseInt(octal, 8)),
);
}
/** Linux exposes the effective options of every mount, including a container's tmpfs mounts. A
* verifier cannot infer `noexec` from file permissions: native addons are mapped executable rather
* than invoked as files. On systems without mountinfo, the load remains the final authority. */
async function isNoExecMount(directory: string): Promise<boolean> {
if (process.platform !== "linux") return false;
let mountinfo: string;
let canonicalDirectory: string;
try {
[mountinfo, canonicalDirectory] = await Promise.all([
readFile("/proc/self/mountinfo", "utf8"),
realpath(directory),
]);
} catch {
return false;
}
const mounts = mountinfo
.split("\n")
.map((line) => line.split(" "))
.filter((fields) => fields.length > 6 && fields.includes("-"))
.map((fields) => ({
mountPoint: decodedMountPath(fields[4] ?? ""),
options: (fields[5] ?? "").split(","),
}))
.filter(
({ mountPoint }) =>
canonicalDirectory === mountPoint ||
canonicalDirectory.startsWith(mountPoint === "/" ? "/" : `${mountPoint}/`),
)
.sort((left, right) => right.mountPoint.length - left.mountPoint.length);
return mounts[0]?.options.includes("noexec") ?? false;
}
async function createExecutionScratch(baseDirectory: string): Promise<string> {
if (!path.isAbsolute(baseDirectory)) {
fail("MODULE_PACKAGE_VERIFIER_SCRATCH_UNAVAILABLE");
}
try {
if (await isNoExecMount(baseDirectory)) {
fail("MODULE_PACKAGE_VERIFIER_SCRATCH_UNAVAILABLE");
}
return await mkdtemp(path.join(baseDirectory, "ekklesia-package-verify-"));
} catch (error) {
if (error instanceof ModulePackageVerificationError) throw error;
fail("MODULE_PACKAGE_VERIFIER_SCRATCH_UNAVAILABLE");
}
}
/** Loads every backend entrypoint in a short-lived child with an empty environment, and reports /** Loads every backend entrypoint in a short-lived child with an empty environment, and reports
* only whether each loaded and exported a definition naming this module. */ * only whether each loaded and exported a definition naming this module. */
async function loadEntrypoints( async function loadEntrypoints(
manifest: Pick<ModuleManifest, "database" | "id" | "version">, manifest: Pick<ModuleManifest, "database" | "id" | "version">,
entrypoints: Readonly<Record<string, string>>, entrypoints: Readonly<Record<string, string>>,
timeoutMs: number, timeoutMs: number,
executionDirectory: string,
): Promise<void> { ): Promise<void> {
const outcome = await new Promise<"ok" | "failed">((resolve) => { const outcome = await new Promise<"ok" | "failed">((resolve) => {
let settled = false; let settled = false;
@@ -445,7 +508,7 @@ async function loadEntrypoints(
}), }),
], ],
{ {
cwd: tmpdir(), cwd: executionDirectory,
env: {}, env: {},
execArgv: [], execArgv: [],
serialization: "json", serialization: "json",
@@ -514,7 +577,7 @@ export async function verifyModulePackage(
const manifest = checkIdentity(archive, metadata, files); const manifest = checkIdentity(archive, metadata, files);
checkCompatibility(metadata, manifest, files, options.platform); checkCompatibility(metadata, manifest, files, options.platform);
const scratch = await mkdtemp(path.join(tmpdir(), "ekklesia-package-verify-")); const scratch = await createExecutionScratch(options.executionScratchDirectory ?? tmpdir());
try { try {
// The candidate alone, in a scratch inventory of its own: never module storage. // The candidate alone, in a scratch inventory of its own: never module storage.
const modulesDirectory = path.join(scratch, "candidates"); const modulesDirectory = path.join(scratch, "candidates");
@@ -550,7 +613,7 @@ export async function verifyModulePackage(
// Executed only now, and first in isolation: a schema is imported in this process below only // Executed only now, and first in isolation: a schema is imported in this process below only
// after the probe proved it loads on its own. // after the probe proved it loads on its own.
await loadEntrypoints(manifest, entrypoints, options.loadTimeoutMs ?? 30_000); await loadEntrypoints(manifest, entrypoints, options.loadTimeoutMs ?? 30_000, scratch);
try { try {
await discoverModuleSchemas(discoveryOptions); await discoverModuleSchemas(discoveryOptions);
} catch (error) { } catch (error) {
+53 -1
View File
@@ -1,4 +1,4 @@
import { mkdtemp, readdir, rm, stat, writeFile } from "node:fs/promises"; import { mkdir, mkdtemp, readdir, rm, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import path from "node:path"; import path from "node:path";
import { afterEach, describe, expect, it } from "vitest"; import { afterEach, describe, expect, it } from "vitest";
@@ -48,6 +48,12 @@ async function archiveFile(options: PackageFixtureOptions = {}): Promise<{
return { archivePath, sha256: fixture.sha256 }; return { archivePath, sha256: fixture.sha256 };
} }
async function executionScratch(): Promise<string> {
const directory = path.join(await scratch(), "execution");
await mkdir(directory);
return directory;
}
async function rejection( async function rejection(
options: PackageFixtureOptions, options: PackageFixtureOptions,
code: ModulePackageVerificationErrorCode, code: ModulePackageVerificationErrorCode,
@@ -100,6 +106,52 @@ describe("Module Package v1 verification (ADR 0049)", () => {
}); });
}); });
it("uses an explicit execution scratch for pure JavaScript and cleans concurrent runs", async () => {
const { archivePath } = await archiveFile();
const executionScratchDirectory = await executionScratch();
const runs = Array.from({ length: 4 }, () =>
verifyModulePackage({ archivePath, executionScratchDirectory, platform }),
);
await expect(Promise.all(runs)).resolves.toHaveLength(4);
expect(await readdir(executionScratchDirectory)).toEqual([]);
});
it("cleans its execution scratch after a backend load failure", async () => {
const { archivePath } = await archiveFile({
serverSource:
'import "dependency-not-in-this-package";\nexport default { id: "example", version: "0.1.0" };\n',
});
const executionScratchDirectory = await executionScratch();
await expect(
verifyModulePackage({ archivePath, executionScratchDirectory, platform }),
).rejects.toMatchObject({ code: "MODULE_PACKAGE_ENTRYPOINT_LOAD_FAILED" });
expect(await readdir(executionScratchDirectory)).toEqual([]);
});
it("reports a missing execution scratch as a verifier environment error", async () => {
const { archivePath } = await archiveFile();
const executionScratchDirectory = path.join(await scratch(), "missing");
await expect(
verifyModulePackage({ archivePath, executionScratchDirectory, platform }),
).rejects.toMatchObject({
code: "MODULE_PACKAGE_VERIFIER_SCRATCH_UNAVAILABLE",
message:
"The verifier execution scratch directory is missing, not writable, or mounted noexec.",
});
await expect(stat(executionScratchDirectory)).rejects.toMatchObject({ code: "ENOENT" });
});
it("rejects an unsafe archive before consulting the execution scratch", async () => {
const { archivePath } = await archiveFile(
withEntry({ path: "example/../attacker-chosen", data: "x" }),
);
const executionScratchDirectory = path.join(await scratch(), "missing");
await expect(
verifyModulePackage({ archivePath, executionScratchDirectory, platform }),
).rejects.toMatchObject({ code: "MODULE_PACKAGE_UNSAFE_PATH" });
await expect(stat(executionScratchDirectory)).rejects.toMatchObject({ code: "ENOENT" });
});
it("extracts only a verified package, with normalized modes, into a new directory", async () => { it("extracts only a verified package, with normalized modes, into a new directory", async () => {
const { archivePath } = await archiveFile(); const { archivePath } = await archiveFile();
const target = path.join(await scratch(), "example"); const target = path.join(await scratch(), "example");
+12
View File
@@ -1,5 +1,6 @@
import { import {
EMPTY_MODULE_STORAGE, EMPTY_MODULE_STORAGE,
isModuleStorageError,
isValidModuleStorageKey, isValidModuleStorageKey,
MODULE_STORAGE_KEY_MAX_BYTES, MODULE_STORAGE_KEY_MAX_BYTES,
MODULE_STORAGE_READ_URL_DEFAULT_SECONDS, MODULE_STORAGE_READ_URL_DEFAULT_SECONDS,
@@ -322,6 +323,17 @@ describe("module storage scoping", () => {
} }
}); });
it("recognizes a storage error from an installed module's separate package copy", () => {
const external = Object.assign(new Error("The storage service could not be reached."), {
code: "STORAGE_PROVIDER_UNAVAILABLE",
name: "ModuleStorageError",
});
expect(external).not.toBeInstanceOf(ModuleStorageError);
expect(isModuleStorageError(external)).toBe(true);
expect(isModuleStorageError(Object.assign(new Error("no"), { code: "UNKNOWN" }))).toBe(false);
});
it("gives a module no way to reach another module's storage by knowing its name", async () => { it("gives a module no way to reach another module's storage by knowing its name", async () => {
const shared = fakeProvider(); const shared = fakeProvider();
const announcements = createModuleStorage({ const announcements = createModuleStorage({
@@ -3,6 +3,8 @@ import path from "node:path";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { import {
MODULE_VERIFIER_SCRATCH_ENV,
moduleVerifierScratchDirectory,
parseSha256File, parseSha256File,
parseVerifyModulePackageArguments, parseVerifyModulePackageArguments,
readModulePackagePlatform, readModulePackagePlatform,
@@ -57,6 +59,21 @@ describe("verify-module-package digest files", () => {
}); });
}); });
describe("verify-module-package execution scratch", () => {
it("accepts an absolute operator-configured directory and otherwise leaves the library default", () => {
expect(moduleVerifierScratchDirectory({})).toBeUndefined();
expect(
moduleVerifierScratchDirectory({ [MODULE_VERIFIER_SCRATCH_ENV]: "/run/ekklesia-verifier" }),
).toBe("/run/ekklesia-verifier");
});
it.each(["", "relative/path"])("refuses the invalid configured path %j", (configured) => {
expect(() =>
moduleVerifierScratchDirectory({ [MODULE_VERIFIER_SCRATCH_ENV]: configured }),
).toThrow(`${MODULE_VERIFIER_SCRATCH_ENV} must be an absolute path.`);
});
});
describe("verify-module-package platform facts", () => { describe("verify-module-package platform facts", () => {
it("reads this platform's versions and its transitional database runtime from the repository", async () => { it("reads this platform's versions and its transitional database runtime from the repository", async () => {
const root = JSON.parse(await readFile(path.join(repositoryRoot, "package.json"), "utf8")); const root = JSON.parse(await readFile(path.join(repositoryRoot, "package.json"), "utf8"));
+15 -3
View File
@@ -17,6 +17,7 @@ specific to one organization.
| [`scripts/init-tls.sh`](scripts/init-tls.sh) | Creates the installation-private CA and the `postgres` and `object-storage` server certificates. | | [`scripts/init-tls.sh`](scripts/init-tls.sh) | Creates the installation-private CA and the `postgres` and `object-storage` server certificates. |
| [`scripts/gen-secrets.sh`](scripts/gen-secrets.sh) | Prints freshly generated secrets to paste into the env files. Writes nothing. | | [`scripts/gen-secrets.sh`](scripts/gen-secrets.sh) | Prints freshly generated secrets to paste into the env files. Writes nothing. |
| [`Dockerfile`](Dockerfile) | The one production image, for the API and every operator command ([The production image](#the-production-image)). | | [`Dockerfile`](Dockerfile) | The one production image, for the API and every operator command ([The production image](#the-production-image)). |
| [`display-kiosk.md`](display-kiosk.md) | Setting up an unattended display screen: launch, pairing, and recovery. |
## The reference deployment ## The reference deployment
@@ -170,9 +171,20 @@ identity, target, and Node.js range against this platform, runs the platform's m
validation, and loads the backend once in a short-lived process with no environment. Only then does validation, and loads the backend once in a short-lived process with no environment. Only then does
it write the verified module directory, to a new directory outside module storage: it write the verified module directory, to a new directory outside module storage:
The backend probe executes the package's server-side code; the verifier container is its isolation
boundary. Keep the general-purpose `/tmp` mount `noexec`, and give only this one-shot verifier a
separate executable scratch tmpfs. The mount below is ephemeral, bounded at 1100 MiB (just above the
package format's 1 GiB expanded limit), `nodev`, `nosuid`, and writable only by the image's
unprivileged `node` user. `EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR` tells the CLI where that mount is;
the verifier creates a unique child for each run and removes it on success or failure.
```sh ```sh
mkdir -m 777 staging # writable by the image's unprivileged user; nothing else is mkdir -m 777 staging # writable by the image's unprivileged user; nothing else is
docker run --rm --network none --read-only --tmpfs /tmp \ docker run --rm --network none --read-only \
--tmpfs /tmp:rw,nosuid,nodev,noexec,size=64m \
--tmpfs /run/ekklesia-verifier:rw,nosuid,nodev,exec,size=1100m,mode=0700,uid=1000,gid=1000 \
--cap-drop ALL --security-opt no-new-privileges \
-e EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR=/run/ekklesia-verifier \
-v "$PWD:/package:ro" -v "$PWD/staging:/staging" \ -v "$PWD:/package:ro" -v "$PWD/staging:/staging" \
--entrypoint node "$EKKLESIA_IMAGE" core/dist/cli/verify-module-package.js \ --entrypoint node "$EKKLESIA_IMAGE" core/dist/cli/verify-module-package.js \
/package/<id>-<version>-any.tar.gz --sha256-file /package/<id>-<version>-any.tar.gz.sha256 \ /package/<id>-<version>-any.tar.gz --sha256-file /package/<id>-<version>-any.tar.gz.sha256 \
@@ -500,10 +512,10 @@ finds its files:
```sh ```sh
docker build \ docker build \
--file deploy/Dockerfile \ --file deploy/Dockerfile \
--build-arg VERSION=0.2.1 \ --build-arg VERSION=0.2.4 \
--build-arg REVISION="$(git rev-parse HEAD)" \ --build-arg REVISION="$(git rev-parse HEAD)" \
--build-arg SOURCE=https://git.example.org/your-org/ekklesia \ --build-arg SOURCE=https://git.example.org/your-org/ekklesia \
--tag ekklesia:0.2.1 \ --tag ekklesia:0.2.4 \
. .
``` ```
+4
View File
@@ -132,6 +132,10 @@ services:
command: command:
- mini - mini
- -dir=/data - -dir=/data
# A bucket is a SeaweedFS collection and claims a block of volumes. The ~1 GB automatic
# volume size leaves modest hosts with room for only a few collections; after that, writes
# stall instead of failing. This changes allocation granularity, not object or total capacity.
- -master.volumeSizeLimitMB=32
- -s3.port.https=8443 - -s3.port.https=8443
- -s3.cert.file=/run/ekklesia/tls/object-storage.crt - -s3.cert.file=/run/ekklesia/tls/object-storage.crt
- -s3.key.file=/run/ekklesia/tls/object-storage.key - -s3.key.file=/run/ekklesia/tls/object-storage.key
+129
View File
@@ -0,0 +1,129 @@
# Setting up a display kiosk
A display is a screen — a television with a small computer, or a mini PC on a wall — that opens one
module's display page full screen and keeps it open unattended. This guide sets one up so it starts
on boot, stays paired, recovers from network and server outages on its own, and never opens a second
copy of itself.
Pairing and display identity belong to the platform (ADR 0036). Today an administrator approves a
screen in the Dashboard sidebar's **Display devices** panel; a later release moves this to
**Settings → Displays**. Nothing below changes when that happens.
## What the screen needs
- Chromium or Google Chrome, current stable. The display page is tested against Chromium.
- A **dedicated, persistent browser profile directory**. The device credential is an HttpOnly cookie
in that profile; it is what keeps the screen paired across reboots. Never use a guest, incognito, or
"clear data on exit" profile, and never share the profile with a person's browsing.
- Network access to the installation's **API hostname** (the display page, its realtime stream, and
its module requests) and to the storage hostname that serves images.
- A clock that is roughly right. Playback follows the server's clock, but TLS does not tolerate a
wildly wrong one.
## The address
Every display-capable module has one display page on the API hostname:
```text
https://<api-hostname>/display/<module-id>
```
For Signage that is `https://<api-hostname>/display/signage`. The Signage **Displays** tab shows the
exact address under **Copy setup address**. Open it on the screen itself; opening it on an
administrator's own computer only previews the display (and that browser appears in the roster as a
*Signed-in browser*, which can be removed once it is offline).
## Launching the browser
Run exactly **one** launcher, and launch the page as an app window with a fixed profile:
```bash
chromium \
--user-data-dir=/var/lib/display-kiosk/profile \
--app=https://<api-hostname>/display/signage \
--kiosk \
--no-first-run \
--noerrdialogs \
--disable-session-crashed-bubble \
--hide-crash-restore-bubble \
--disable-features=Translate \
--autoplay-policy=no-user-gesture-required \
--check-for-update-interval=31536000
```
- `--app` opens the page with no tab strip and no address bar. `--kiosk` makes it full screen.
- `--user-data-dir` is the persistent profile that holds the pairing. Keep it on persistent storage
and back it up with nothing: if it is lost, the screen simply shows a new pairing code.
- Do **not** add `--restore-last-session`, and do not also start the browser from a desktop autostart
entry. Chromium allows one process per profile; a second launch hands its URL to the running
process, which opens a **second window or tab** of the display. That is the usual cause of
"duplicate" displays. One launcher, one profile, one window.
### Start on boot and restart on exit (systemd)
Run the browser as an unprivileged user in that user's graphical session, from a single user unit:
```ini
# ~/.config/systemd/user/display-kiosk.service
[Unit]
Description=Display kiosk
After=graphical-session.target network-online.target
Wants=network-online.target
PartOf=graphical-session.target
[Service]
ExecStart=/usr/bin/chromium --user-data-dir=%h/.local/share/display-kiosk --app=https://<api-hostname>/display/signage --kiosk --no-first-run --noerrdialogs --disable-session-crashed-bubble --hide-crash-restore-bubble --disable-features=Translate --autoplay-policy=no-user-gesture-required --check-for-update-interval=31536000
Restart=always
RestartSec=5
[Install]
WantedBy=graphical-session.target
```
```bash
systemctl --user daemon-reload
systemctl --user enable --now display-kiosk.service
loginctl enable-linger <kiosk-user> # with the desktop set to log this user in automatically
```
`systemctl --user stop display-kiosk` closes the browser cleanly, which is what keeps its profile —
and the pairing cookie — intact. Avoid cutting power to a running kiosk when you can: a clean shutdown
is always safer for a browser profile.
## Pairing
1. Start the kiosk. With no credential it shows a **six-digit pairing code**.
2. In the Dashboard, enter the code under **Display devices**, give the screen a name, and approve.
3. Within a few seconds the screen collects its credential and starts showing content.
4. In Signage → **Displays**, name the display and choose its group and audience.
A paired screen stays paired across browser restarts and reboots. The credential lasts **180 days**
from pairing and is not renewed automatically (ADR 0036); after that the screen returns to its pairing
code and is paired again the same way. Note the pairing date of each screen.
## What happens when things go wrong
| Event | What the screen does |
| --- | --- |
| Network drops for seconds | Nothing visible. The display keeps running; the realtime stream reconnects. |
| Network or API down for longer | Keeps playing what it has cached, with a small corner note; recovers by itself when the server is back. No remount, no reload. |
| API restarted or upgraded | Reconnects by itself. A new module version is picked up within five minutes and swapped in. |
| Browser crashes or is restarted | systemd restarts it; the same profile is still paired. |
| Machine reboots | Starts on boot and resumes. If the server is not up yet, the screen retries on its own (every few seconds, backing off to once a minute) until it is. |
| Display revoked in the Dashboard | Stops showing content within seconds and returns to its pairing code. |
| Display moved to another group or audience | Switches content within about one heartbeat (≈12 s), no reload. |
**Offline cold start is not supported yet.** A screen that is already showing content survives an
outage. A screen that *boots* while the server is completely unreachable can load its cached page but
cannot mount the module until the server answers, because the authorization it needs is never cached
(ADR 0043, Signage ADR 0006). It shows a short message and retries on its own until the server is
reachable — nobody needs to touch it.
## Checklist for a new kiosk
- [ ] One systemd unit, no desktop autostart entry, no second launcher.
- [ ] `--user-data-dir` on persistent storage, not shared, not cleared on exit.
- [ ] Automatic login of the kiosk user, and screen blanking/power saving disabled.
- [ ] Paired and named; assigned in Signage; pairing date recorded.
- [ ] Unplug the network for a minute: content keeps playing, then the note clears on reconnect.
- [ ] Reboot: the screen comes back paired without anyone touching it.
@@ -1,6 +1,6 @@
# 0052: Reviewed license supplements cover redistributed dependencies whose packages omit required material # 0052: Reviewed license supplements cover redistributed dependencies whose packages omit required material
- Status: Proposed - Status: Accepted
- Date: 2026-10-07 - Date: 2026-10-07
- Extends: [0049](0049-module-package-v1.md), in one respect only. ADR 0049 left exactly how licence - Extends: [0049](0049-module-package-v1.md), in one respect only. ADR 0049 left exactly how licence
and notice material is collected to the packager's implementation. That implementation, and the and notice material is collected to the packager's implementation. That implementation, and the
@@ -9,7 +9,9 @@
further source: a reviewed supplement. Everything else in ADR 0049 is unchanged, as is further source: a reviewed supplement. Everything else in ADR 0049 is unchanged, as is
`packageFormat`. `packageFormat`.
- Related: [ADR 0048](0048-generic-image-and-installed-module-storage.md), - Related: [ADR 0048](0048-generic-image-and-installed-module-storage.md),
[ADR 0051](0051-module-package-license-material.md) [ADR 0051](0051-module-package-license-material.md),
[ADR 0055](0055-corresponding-source-artifacts.md) (decides the source-obligation policy deferred
below)
## Context ## Context
@@ -0,0 +1,189 @@
# 0055: Native redistributable packages carry corresponding-source artifacts
- Status: Accepted
- Date: 2026-10-09
- Extends: [0049](0049-module-package-v1.md). ADR 0049 withholds distributable-release status from a
package until its licence handling is proven. This ADR adds what proving it means for a native
package that redistributes copyleft code: a sibling corresponding-source artifact, and a
fail-closed gate for public distribution. Module Package v1 is unchanged, and `packageFormat`
stays **1**.
- Decides: the item [ADR 0052](0052-reviewed-license-supplements.md) deferred, "the legal and
compliance determination of source obligations for each redistributed binary, and how a
determination is recorded", as a project policy. Reviewed supplements are unchanged in purpose
and format.
- Related: [ADR 0051](0051-module-package-license-material.md), whose rules for a module's own
licence are unchanged.
## Context
A native Module Package (ADR 0049) copies native dependencies into its `node_modules` exactly as
built. The first one, the packager's native proof with sharp, redistributes
`@img/sharp-libvips-linux-x64`. That is one shared object with about 29 libraries built into it.
Several are under the GNU LGPL, and cairo and some Rust crates are under the MPL.
ADR 0052 made the licence and notice side work. A reviewed supplement adds the licence, copyright,
and notice files the package omits, with the exact upstream source each came from. But ADR 0052 was
explicit that carrying material is not meeting obligations. The LGPL and the MPL also oblige whoever
distributes the binary to make the corresponding source available. For a library linked into a
larger work, that includes what a recipient needs to rebuild the library and use a modified version.
Recording upstream URLs does not do that: URLs move, tags are rewritten, and a mirror is somebody
else's promise.
Until this was decided, every native package containing such components was blocked from public
distribution, and the blocker had no defined way to be lifted.
## Decision
### Native packages may contain copyleft native dependencies
A native Module Package may redistribute native dependencies under the LGPL, the MPL, or a similar
licence with source obligations. It may be **distributed publicly** only when it is released
together with a **corresponding-source artifact** that meets this ADR. A package whose artifact
cannot be produced and verified is not approved for public distribution.
This is Ekklesia's **project compliance policy**: a conservative rule the project holds itself to.
It is not legal advice, and it is not legal certification. Repository review of the artifact, its
inputs, and its instructions is project and code review. It does not replace professional legal
review, and nothing built from this ADR may claim that it does.
### Supplements and source artifacts do different jobs
- **A reviewed supplement (ADR 0052)** identifies and verifies the licence and notice material for a
redistributed binary. It travels inside the package, in `THIRD_PARTY_NOTICES`.
- **A corresponding-source artifact** carries the source, patches, build definitions, and rebuild
and relink instructions for the covered native components. It travels beside the package, as a
separate release artifact.
A supplement alone never satisfies the source-availability policy. A source artifact never replaces
the supplement or `THIRD_PARTY_NOTICES`.
### What the artifact contains
For each covered native dependency the package redistributes, the artifact contains:
- the exact source the binary was built from, as bytes, not as references:
- its build definitions;
- the source archive of every library compiled into it;
- every patch the build applied;
- every source package a language package manager resolved for the build;
- the source of any runtime library the toolchain linked into it;
- the licence and copyright material: the reviewed supplement, and the files inside the sources;
- **instructions** that are technically actionable. They name what the artifact corresponds to and
its target, and say how to apply the patches and rebuild the covered library. They also say how a
rebuilt or modified library replaces the redistributed one in the installed module, and which
toolchain and configuration to use, or point to the included build definitions;
- a deterministic, machine-readable manifest of every component and file, each with its digest,
plus the human-readable provenance.
Components that are not part of the binary are left out, each with its reason recorded: build-only,
test-only, or another platform's. The exception is a build-time input needed to rebuild the covered
library, such as a source package a build script requires. That is included and marked as
build-only. Host system libraries and general-purpose build tools are not redistributed, and the
instructions name the versions they need.
### It is tied to exact inputs, and verified before it is written
- **Bound to the binary.** An artifact names each covered dependency by package name, version, and
the content digest a supplement is matched by (ADR 0052). It names the module identity and target
of the package it accompanies. An artifact for one version of a binary never satisfies another.
- **From reviewed inputs only.** The inputs come from a checked-in, reviewed **source recipe** per
covered dependency version. The recipe records each component's name, version, origin,
relationship to the binary, and licence material, and each source file's and patch's size and
SHA-256.
- **Acquisition is separate from construction.** Obtaining the recorded bytes may use the network,
in an explicit preparation step that stores them in a local cache keyed by content digest.
Constructing the artifact is offline. It reads only the cache and checked-in files, verifies every
byte against the recipe before writing anything, and never fetches or trusts a current upstream
response.
- **Deterministic.** The same inputs produce the same artifact, byte for byte, in any working
directory: normalized relative paths, fixed ownership, modes, and timestamps, sorted entries, and
no machine-specific path.
### Binary and source are linked by recorded digests
A public release of a native package with covered components consists of sibling artifacts:
- the Module Package archive and its SHA-256 (ADR 0049);
- the corresponding-source archive and its SHA-256;
- machine-readable **redistribution metadata** recording:
- the package archive's name and SHA-256;
- the module identity and target;
- each covered dependency's name, version, licence, and content digest;
- the supplement and recipe it was assembled from;
- the source archive's name and SHA-256, and the SHA-256 of its manifest;
- a **release notice**, displayed alongside the binary wherever it is offered. It says, in
substance, that the release includes third-party software under licences including the GNU LGPL
or the MPL, and that corresponding source and build and relink materials for the covered native
dependencies are in the accompanying artifact, named by its exact file name and SHA-256.
A reviewer can therefore tell exactly which source artifact corresponds to which redistributed
binary, from digests, without guessing from file names. The source archive is never placed inside
the Module Package. `THIRD_PARTY_NOTICES` stays as it is, and never duplicates the source.
### Public distribution fails closed
Building and testing packages stays unrestricted: the packager produces native packages exactly as
before, for development, CI, and private use. **Public redistribution approval** is a separate gate.
For a package intended for public distribution, the gate requires that:
- the package's native dependency inventory is known;
- every covered dependency has its reviewed supplement and source recipe;
- the corresponding-source artifact is present;
- every digest matches;
- the artifact's manifest validates against the recipe and the archive's own contents;
- the release notice names the exact artifact.
If any requirement fails, approval fails. Bundled (not copied) code under such a licence is not
handled by this mechanism, so the gate refuses it rather than approving it.
### Retention, and no rewriting history
- Ekklesia keeps a release's corresponding-source artifact for as long as it distributes, or
supports access to, the matching binary release. Nothing deletes one automatically while that
binary remains available.
- The bytes of a source artifact for an existing binary release are never replaced under the same
identity.
- A correction is a new release, or an explicitly versioned compliance artifact that the
redistribution metadata names. History is never silently replaced.
### What this ADR does not do
- It does not change any module's own licence, or ADR 0051's rules for it. The module's own source
obligations are met by the module's own repository, as before.
- It does not make **runtime downloading** an acceptable workaround. Fetching a copyleft library at
run time instead of redistributing it moves the obligation rather than meeting it, makes
installations depend on a third party, and breaks ADR 0048's self-contained installed modules. It
is not the preferred path.
- It does not change Module Package v1, the verifier's acceptance of a package, or the content of
existing supplements.
## Consequences
- Native modules that need copyleft native libraries, starting with sharp's libvips, can be
distributed publicly under a defined, checkable policy, instead of being blocked indefinitely.
- Each covered dependency version needs a reviewed source recipe, as it needs a reviewed supplement.
A dependency bump needs both before public distribution works again.
- Releases carry an artifact many times larger than the package itself (about 200 MB for libvips
1.3.4, against a 9 MB package), and Ekklesia commits to retaining it.
- Constructing the artifact needs the verified source cache. Acquisition happens once per recipe,
not on every build.
## Alternatives
- **Treat supplements, source URLs, or a written offer as sufficient.** Rejected: URLs and offers
depend on third parties and on time. This policy requires the source itself, retained with the
release.
- **Put the source inside the Module Package.** Rejected: installations would store hundreds of
megabytes they never need, and the package format would change for a distribution concern.
- **Download sources while constructing the artifact.** Rejected: a release would depend on mutable
remote responses. Acquisition is explicit and verified, and construction is offline.
- **Download native libraries at run time.** Rejected, as above.
- **Per-dependency special cases in code.** Rejected for the same reason as in ADR 0052: recipes are
reviewed data, and the mechanism knows no particular library.
## Deferred
- Hosting and serving source artifacts for published module releases, and the release CI that runs
the gate and keeps the verified source cache.
- Corresponding source for bundled copyleft code, if a module ever needs it.
- Recipes provided by module repositories, alongside the question ADR 0052 deferred for supplements.
+2 -1
View File
@@ -60,6 +60,7 @@ ADRs capture durable choices that affect more than one package or phase.
- [0049: A module release is one immutable, self-contained package: Ekklesia Module Package v1](0049-module-package-v1.md) — Accepted - [0049: A module release is one immutable, self-contained package: Ekklesia Module Package v1](0049-module-package-v1.md) — Accepted
- [0050: Module Database ABI v1: modules and Core exchange SQL, never ORM objects](0050-module-database-abi-v1.md) — Accepted - [0050: Module Database ABI v1: modules and Core exchange SQL, never ORM objects](0050-module-database-abi-v1.md) — Accepted
- [0051: Module packages declare their own license and carry its text](0051-module-package-license-material.md) — Accepted - [0051: Module packages declare their own license and carry its text](0051-module-package-license-material.md) — Accepted
- [0052: Reviewed license supplements cover redistributed dependencies whose packages omit required material](0052-reviewed-license-supplements.md) — Proposed - [0052: Reviewed license supplements cover redistributed dependencies whose packages omit required material](0052-reviewed-license-supplements.md) — Accepted
- [0053: A module builds outside the monorepo, from Ekklesia packages published over the npm registry protocol](0053-standalone-module-development-packages.md) — Accepted - [0053: A module builds outside the monorepo, from Ekklesia packages published over the npm registry protocol](0053-standalone-module-development-packages.md) — Accepted
- [0054: Ekklesia's module-development packages are publicly readable before launch](0054-public-package-reads-before-launch.md) — Accepted - [0054: Ekklesia's module-development packages are publicly readable before launch](0054-public-package-reads-before-launch.md) — Accepted
- [0055: Native redistributable packages carry corresponding-source artifacts](0055-corresponding-source-artifacts.md) — Accepted
+10 -1
View File
@@ -692,10 +692,19 @@ Module Package v1, a deterministic `.tar.gz` with its SHA-256.
reviewed supplements checked in with the packager add it reviewed supplements checked in with the packager add it
([ADR 0052](decisions/0052-reviewed-license-supplements.md)). Each supplement matches one exact ([ADR 0052](decisions/0052-reviewed-license-supplements.md)). Each supplement matches one exact
package name, version, and content digest; anything else fails closed. package name, version, and content digest; anything else fails closed.
- **Corresponding source.** A native package that redistributes code under the LGPL, the MPL, or
a similar licence is approved for public distribution only together with a sibling
corresponding-source artifact. The artifact is built offline from a reviewed source recipe and a
verified source cache, and linked to the package by digest
([ADR 0055](decisions/0055-corresponding-source-artifacts.md)). Building and testing packages is
unaffected.
- **The verifier.** Core's `verifyModulePackage` reads the archive in memory and refuses links, - **The verifier.** Core's `verifyModulePackage` reads the archive in memory and refuses links,
special files, and escaping paths. It checks every digest and identity, the target and runtime, special files, and escaping paths. It checks every digest and identity, the target and runtime,
runs the platform's own module, schema, and migration validation, and loads the backend once in a runs the platform's own module, schema, and migration validation, and loads the backend once in a
short-lived process with an empty environment. Only then does it extract anything. short-lived process with an empty environment. Only then does it copy the package to a requested
output directory. In the hardened container invocation, `/tmp` remains `noexec`; the temporary
candidate lives on a dedicated, bounded executable tmpfs because a native addon's code must be
mapped executable to load.
- **A native package's host checks.** It is accepted only on a host whose operating system, CPU, - **A native package's host checks.** It is accepted only on a host whose operating system, CPU,
C library family and version, and Node-API version Node.js itself reports as compatible. C library family and version, and Node-API version Node.js itself reports as compatible.
- **ELF files.** Every ELF file in it must be built for the target's machine. - **ELF files.** Every ELF file in it must be built for the target's machine.
+7 -1
View File
@@ -484,7 +484,13 @@ before it is installed: its digest, its archive safety (no links, special files,
escaping paths), every file's digest, and its identity and compatibility. The digest proves escaping paths), every file's digest, and its identity and compatibility. The digest proves
integrity, not who published the package; signing is not implemented. The verifier loads the backend integrity, not who published the package; signing is not implemented. The verifier loads the backend
in a short-lived process with an empty environment, so module code never sees a database URL or a in a short-lived process with an empty environment, so module code never sees a database URL or a
secret while it is being checked. secret while it is being checked. That load executes package code: `noexec` is retained on the
general `/tmp`, but native packages require the verifier's narrow, bounded execution scratch to be
executable. The one-shot hardened verifier container—not the scratch mount—is the isolation
boundary, and it is not claimed to be a perfect sandbox. Use the supported invocation in
[`deploy/README.md`](../../deploy/README.md#installed-modules), which also keeps the root filesystem
read-only, runs as the image's non-root user, disables networking and privilege escalation, and
drops Linux capabilities.
Two production defaults are relevant but are **not** by themselves a deployment guarantee: Two production defaults are relevant but are **not** by themselves a deployment guarantee:
+105 -4
View File
@@ -296,9 +296,10 @@ The archive holds one directory named with your module id:
- **What it does not do.** A supplement never changes the licence the package declares. The - **What it does not do.** A supplement never changes the licence the package declares. The
supplements cover `@img/sharp-libvips-linux-x64` 1.3.3 and 1.3.4, the libvips binaries that supplements cover `@img/sharp-libvips-linux-x64` 1.3.3 and 1.3.4, the libvips binaries that
`sharp` 0.35.4 and 0.35.5 load; each matches only its own version and digest, and the packager's `sharp` 0.35.4 and 0.35.5 load; each matches only its own version and digest, and the packager's
native proof builds with sharp 0.35.5. They record upstream sources, but whether the LGPL and MPL native proof builds with sharp 0.35.5. A supplement identifies and verifies licence and notice
source obligations for those binaries are met is an open determination. A package that carries material only. Source availability for copyleft native code is a separate requirement, met by a
one is not yet treated as publicly distributable. corresponding-source artifact
([Distributing a native package publicly](#distributing-a-native-package-publicly)).
- **Deterministic.** The same packager version and the same built inputs produce the same archive, - **Deterministic.** The same packager version and the same built inputs produce the same archive,
byte for byte, wherever the checkout is. byte for byte, wherever the checkout is.
@@ -317,13 +318,113 @@ require exact agreement with Core. The legacy facade has not been removed.
`node core/dist/cli/verify-module-package.js <archive> --sha256-file <archive>.sha256`, run from an `node core/dist/cli/verify-module-package.js <archive> --sha256-file <archive>.sha256`, run from an
Ekklesia installation (`/opt/ekklesia` in the image), checks a package against that platform without Ekklesia installation (`/opt/ekklesia` in the image), checks a package against that platform without
installing it. Pass `--extract-to <new directory>` to receive the verified module directory. installing it. Pass `--extract-to <new directory>` to receive the verified module directory.
[deploy/README.md](../../deploy/README.md#installed-modules) shows the whole installation. [deploy/README.md](../../deploy/README.md#installed-modules) shows the supported hardened container
invocation; use it rather than inventing mount flags. Backend verification executes package code.
Its container is therefore the isolation boundary: general `/tmp` remains `noexec`, while a narrow,
bounded, ephemeral verifier scratch tmpfs permits execution so native addons can load.
The repository's boundary checks (`pnpm boundaries`) prove only source modules in a workspace. An The repository's boundary checks (`pnpm boundaries`) prove only source modules in a workspace. An
installed package normally has no source, so the platform cannot check it the same way. Your installed package normally has no source, so the platform cannot check it the same way. Your
module's own CI is where its boundary rules, its production dependencies, and its behaviour inside module's own CI is where its boundary rules, its production dependencies, and its behaviour inside
Ekklesia are proven ([ADR 0047](../architecture/decisions/0047-module-ci-harness.md)). Ekklesia are proven ([ADR 0047](../architecture/decisions/0047-module-ci-harness.md)).
#### Distributing a native package publicly
Building and testing a native package needs nothing beyond the packager. **Distributing it
publicly** does, when it redistributes native code under the GNU LGPL, the MPL, or a similar licence
with source obligations
([ADR 0055](../architecture/decisions/0055-corresponding-source-artifacts.md)).
- **Covered dependencies.** A native dependency is covered when its declared licence, or any
component its reviewed supplement records inside it, carries source obligations. libvips is one:
it declares `LGPL-3.0-or-later`, and cairo and several Rust crates inside it are under the MPL.
- **The policy.** A package with covered dependencies may be distributed publicly only together with
a corresponding-source artifact. The artifact holds the exact source, patches, build definitions,
and rebuild and relink instructions for those dependencies, as bytes rather than links. If the
artifact cannot be produced and verified, the package is not approved for public distribution.
This is Ekklesia's project compliance policy, not legal advice or certification. A package that
passes the gate meets Ekklesia's redistribution policy, and nothing more is claimed.
- **Source recipes.** For each covered dependency version, a reviewed recipe is checked in beside
the supplements, under `packages/module-pack/corresponding-sources/<package>/<version>/`. It holds
`recipe.json` and its documents, notably `BUILDING.md`. The recipe names the exact package (name,
version, content digest) and the exact supplement it was prepared with. It lists every component
with its role (`compiled`, `build-input`, or `build-definition`), licence, licence material, and
relationship to the binary. Every source file and patch is recorded by origin, size, and SHA-256,
and every exclusion has a reason. Like supplements, recipes are reviewed data, and the mechanism
knows no particular library.
Releasing a native package publicly takes three steps after packaging:
```bash
# 1. Acquisition: the only step that uses the network. It fetches each recorded file once into a
# cache keyed by SHA-256, refusing any response that differs from the recipe.
pnpm exec ekklesia-corresponding-source acquire --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
--cache ~/.cache/ekklesia/corresponding-sources
# 2. Construction: offline. Every byte is verified against the recipe before anything is written.
pnpm exec ekklesia-corresponding-source build --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
--cache ~/.cache/ekklesia/corresponding-sources --out ../packages
# 3. The public-distribution gate. Exit status 0 means approved.
pnpm exec ekklesia-corresponding-source verify --package ../packages/<id>-<version>-linux-x64-glibc.tar.gz \
--source ../packages/<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz \
--metadata ../packages/<id>-<version>-linux-x64-glibc.redistribution.json
```
Construction writes four sibling release artifacts beside the package. The source archive never goes
inside the Module Package, and `THIRD_PARTY_NOTICES` is unchanged.
```text
<id>-<version>-linux-x64-glibc.tar.gz the Module Package (packager)
<id>-<version>-linux-x64-glibc.tar.gz.sha256
<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz the corresponding source
<id>-<version>-linux-x64-glibc-corresponding-source.tar.gz.sha256
<id>-<version>-linux-x64-glibc.redistribution.json binary ↔ source, by digest
<id>-<version>-linux-x64-glibc.source-notice.txt the release notice
```
- **The source archive** is deterministic: sorted entries, fixed modes, ownership, and timestamps,
and no machine-specific path. The same package and recipe produce the same bytes in any directory.
- `corresponding-source.json` records every file with its SHA-256, and `SHA256SUMS` repeats them.
- Each covered dependency has its own `dependencies/<package>/<version>/` directory, with
`BUILDING.md`, the recipe, `PROVENANCE.md`, the supplement, and `sources/` and `patches/`.
- **The redistribution metadata** answers "which exact source corresponds to this exact binary?"
from digests. It records:
- the package file and its SHA-256, the module identity, and the target;
- each covered dependency's version and content digest, and the supplement and recipe it was
assembled from;
- the source archive's file name and SHA-256, and the SHA-256 of its manifest.
- **The release notice** must be shown alongside the binary wherever it is offered. It states that
the release includes third-party software under licences including the GNU LGPL and the MPL, and
that corresponding source and build and relink materials for the covered native dependencies are
in the accompanying artifact. It names that artifact by its exact file name and SHA-256.
- **Fail-closed.** The gate refuses a package when:
- a covered dependency has no recipe, or a recipe or supplement does not match its exact bytes;
- the source archive is missing, non-canonical, or differs from its manifest;
- the archive holds a file the manifest does not record, or a generated file names a local path;
- the metadata or the notice does not name these exact files.
It also refuses bundled (not copied) copyleft code, which this mechanism does not cover. A package
with no covered native code is approved without an artifact.
- **Retention.** Keep a release's source artifact for as long as its binary release is distributed
or supported. Never delete one automatically while the binary remains available. Never replace its
bytes under the same name. A correction is a new release, or an explicitly versioned compliance
artifact.
**Available today:** a recipe for `@img/sharp-libvips-linux-x64` 1.3.4, used by sharp 0.35.5. Its
artifact is about 200 MB: 27 library archives, 5 patches, 353 Rust crates, and the Rust standard
library source of the recorded toolchain. Following its `BUILDING.md` rebuilds
`libvips-cpp.so.8.18.7` from that source alone. libvips 1.3.3 (sharp 0.35.4) has a supplement but no
recipe, so a package with it builds and tests as before but is not approved for public distribution.
`pnpm test:corresponding-source` runs the whole proof for the native proof module: acquisition into a
verified cache, two byte-identical builds, the gate, and sharp running from the package.
**Adding a native dependency** follows the same path as libvips:
- curate and review its supplement (ADR 0052);
- curate and review its source recipe and `BUILDING.md`, recording every source input the
dependency's build fetches;
- prove both with the native packaging tests and `pnpm test:corresponding-source`.
### License ### License
Ekklesia is licensed under the GNU Affero General Public License, version 3 or (at your option) any Ekklesia is licensed under the GNU Affero General Public License, version 3 or (at your option) any
+2 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "ekklesia", "name": "ekklesia",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"private": true, "private": true,
"type": "module", "type": "module",
@@ -43,6 +43,7 @@
"test:migrations:integration": "pnpm build && bash scripts/test-migrations-integration.sh", "test:migrations:integration": "pnpm build && bash scripts/test-migrations-integration.sh",
"test:root": "vitest run --config vitest.config.ts", "test:root": "vitest run --config vitest.config.ts",
"test:package-release": "bash scripts/test-package-release.sh", "test:package-release": "bash scripts/test-package-release.sh",
"test:corresponding-source": "bash scripts/test-corresponding-source.sh",
"test:package-publication": "bash scripts/test-package-publication.sh", "test:package-publication": "bash scripts/test-package-publication.sh",
"test:watch": "vitest", "test:watch": "vitest",
"typecheck": "turbo run typecheck typecheck:root", "typecheck": "turbo run typecheck typecheck:root",
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/contracts", "name": "@ekklesia/contracts",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
"files": [ "files": [
+22
View File
@@ -87,6 +87,28 @@ export class ModuleStorageError extends Error {
} }
} }
/**
* Whether `value` is a module-storage error, judged by its stable public shape rather than its class.
*
* An installed module bundles its own copy of the contracts package (ADR 0049), so an error created
* by the host's storage provider does not share class identity with the module's
* `ModuleStorageError`. `instanceof ModuleStorageError` is therefore not a valid check across the
* module boundary.
*/
export function isModuleStorageError(
value: unknown,
): value is Pick<ModuleStorageError, "code" | "key" | "message" | "name"> {
if (!(value instanceof Error) || value.name !== "ModuleStorageError") {
return false;
}
const { code, key } = value as { code?: unknown; key?: unknown };
return (
typeof code === "string" &&
(MODULE_STORAGE_ERROR_CODES as readonly string[]).includes(code) &&
(key === undefined || typeof key === "string")
);
}
/** The S3 object-key ceiling, which every S3-compatible provider shares. Measured in UTF-8 bytes, /** The S3 object-key ceiling, which every S3-compatible provider shares. Measured in UTF-8 bytes,
* not characters, because that is what the wire format bounds. */ * not characters, because that is what the wire format bounds. */
export const MODULE_STORAGE_KEY_MAX_BYTES = 1024; export const MODULE_STORAGE_KEY_MAX_BYTES = 1024;
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/design-tokens", "name": "@ekklesia/design-tokens",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
"files": [ "files": [
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/module-database", "name": "@ekklesia/module-database",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
"files": [ "files": [
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env node
// Committed launcher, deliberately not generated: see ekklesia-pack-module.mjs for why. It defers to
// the compiled CLI at run time.
import { fileURLToPath } from "node:url";
const compiledCli = new URL("../dist/corresponding-source-cli.js", import.meta.url);
try {
await import(compiledCli.href);
} catch (error) {
if (error?.code === "ERR_MODULE_NOT_FOUND") {
process.stderr.write(
`@ekklesia/module-pack is not built yet (${fileURLToPath(compiledCli)}).\n` +
"Run `pnpm build` from the repository root first.\n",
);
process.exitCode = 1;
} else {
throw error;
}
}
@@ -0,0 +1,144 @@
# Rebuilding `@img/sharp-libvips-linux-x64` 1.3.4
This directory is the corresponding source for one redistributed native binary. These instructions
say how to rebuild it from the source here, and how to use a rebuilt or modified library in place of
the redistributed one. They are technical instructions. They are not a legal opinion, and they have
not been legally certified.
## What this corresponds to
| | |
| --- | --- |
| Package | `@img/sharp-libvips-linux-x64` 1.3.4, from the npm registry |
| Declared licence | `LGPL-3.0-or-later` |
| Content digest | `ekklesia-content-v1:sha256:2927fe11493461fa34847191d0b9b1c6441007c3e99f306e5708cb07fb0a5e59` |
| Binary | `lib/libvips-cpp.so.8.18.7` (SONAME `libvips-cpp.so.8.18.7`) |
| Target | Linux, x86-64, glibc 2.28 or newer |
| Used by | `sharp` 0.35.5, through `@img/sharp-linux-x64` 0.35.5 |
`lib/libvips-cpp.so.8.18.7` is one shared object. libvips and the libraries it uses are built as
static libraries and linked into it: 27 C and C++ libraries, glib's bundled gvdb, librsvg with its
Rust crates, and the Rust standard library. `recipe.json` lists every component, its licence, its
source file and patches with their SHA-256, and its relationship to the binary. `PROVENANCE.md`
records how all of this was established, and `licenses/` holds the reviewed licence and notice
material.
## What is here
- `sources/sharp-libvips-1.3.4.tar.gz`: the build definitions that produced the binary. They are
`build.sh`, `build/posix.sh`, `platforms/linux-x64/` (a Dockerfile and its toolchain files), and
`versions.properties`.
- `sources/*.tar.*`: the source archive of every compiled library, exactly as `build/posix.sh`
downloads it.
- `patches/`: the five patches `build/posix.sh` applies: glib, mozjpeg, librsvg, libultrahdr, and
libvips.
- `sources/crates/`: all 353 Rust crates librsvg's resolved `Cargo.lock` records.
- 147 are compiled into the binary.
- 17 are needed only to build it: build-script dependencies, and one procedural macro that adds no
code.
- 189 serve librsvg's other workspace members, its tests, or other platforms. Cargo resolves the
whole workspace lockfile even when it builds one package, so it needs them present to build
offline. None of them is compiled into the binary.
`recipe.json` records each crate's role.
- `sources/rust/rust-src-nightly-2026-09-27.tar.xz`: the standard-library source of the Rust toolchain
that built librsvg, `rustc 1.101.0-nightly (75a75c3e0 2026-09-26)`.
- `build/`: the resolved librsvg `Cargo.lock`, and the two scripts below that point the unchanged build
logic at these files.
Every file's SHA-256 is in `recipe.json` and in the archive's top-level `corresponding-source.json`.
Check them before building.
## How the original was built
sharp-libvips builds the binary in a Rocky Linux 8 container, which pins glibc 2.28:
```sh
./build.sh linux-x64
# = docker build --pull -t vips-dev-linux-x64 platforms/linux-x64
# docker run --rm -v "$PWD:/packaging" vips-dev-linux-x64 sh -c /packaging/build/posix.sh
```
`platforms/linux-x64/Dockerfile` installs the toolchain: gcc-toolset-15, cmake, nasm, pkgconf, meson
and ninja (from pip), a nightly Rust toolchain through rustup, and cargo-c. `build/posix.sh` then
downloads each library's archive, applies the patches, builds every library as a static library into
`/target`, and links libvips-cpp. It packs the result as `sharp-libvips-linux-x64.tar.gz`, whose
`lib/libvips-cpp.so.8.18.7` is the file the npm package ships. Meson runs with
`wrap_mode = 'nofallback'` (`platforms/linux-x64/meson.ini`), so it downloads no subproject. The only
forced fallback, gvdb, builds from the copy inside glib's archive.
## Rebuilding from this source
You need a Linux x86-64 host with Docker. Building the container image needs network access to fetch
the toolchain (Rocky Linux packages, rustup, pip, and crates.io for cargo-c). Source for the library
itself comes only from this directory.
1. **Unpack the build definitions.**
```sh
tar -xzf sources/sharp-libvips-1.3.4.tar.gz # creates sharp-libvips-1.3.4/
```
2. **Point the build at this source.**
```sh
sh build/prepare-offline.sh sharp-libvips-1.3.4
```
It copies the archives and patches into `sharp-libvips-1.3.4/offline/`. It unpacks the crates into
a vendored directory source; Cargo checks each against the lockfile's checksum. It then makes three
mechanical edits:
- in `build/posix.sh`, `CURL` becomes `offline/offline-curl.sh`. That script answers each URL the
script would download with its local copy, using `build/url-map.tsv`, and fails for any other URL;
- in `build/posix.sh`, `cargo update --workspace` becomes a copy of the resolved `Cargo.lock`. Cargo
is set offline and uses the vendored crates;
- in `platforms/linux-x64/Dockerfile`, the Rust toolchain is pinned to `nightly-2026-09-27`, the
toolchain whose `rustc` built the original. The upstream file installs whatever nightly is
current.
Nothing that is compiled changes: every patch is still applied by `build/posix.sh`, exactly as
upstream.
3. **Build.**
```sh
cd sharp-libvips-1.3.4
./build.sh linux-x64
```
The result is `sharp-libvips-1.3.4/sharp-libvips-linux-x64.tar.gz`. Its
`lib/libvips-cpp.so.8.18.7` is the rebuilt library. A full build takes from tens of minutes to an
hour, depending on the host.
**Modifying a component.** Before step 3, change the source the build uses. Then `build/posix.sh`
compiles your version in place of the original:
- for a C or C++ library, replace its archive in `offline/files/` under the same name, or add a patch
to the matching step of `build/posix.sh`;
- for a Rust crate, edit it under `offline/vendor/`. The `.cargo-checksum.json` files list no
per-file checksums, so an edited crate still builds.
The result is not byte-identical to the redistributed binary: a compiler toolchain, timestamps, and
paths that were not recorded can all differ. It is built from the same source, by the same
definitions, for the same target.
## Using a rebuilt or modified library
sharp's native addon, `@img/sharp-linux-x64/lib/sharp-linux-x64-0.35.5.node`, links
`libvips-cpp.so.8.18.7` dynamically. Its RPATH begins with `$ORIGIN/../../sharp-libvips-linux-x64/lib`.
In an installed or extracted Ekklesia Module Package, that resolves to:
```text
<module directory>/node_modules/@img/sharp-libvips-linux-x64/lib/libvips-cpp.so.8.18.7
```
To use your library, replace that file with your build's `lib/libvips-cpp.so.8.18.7`. Keep the file
name and SONAME, and keep libvips's C++ API compatible with libvips 8.18, which sharp 0.35.5 is
compiled against. Then restart the process that loads the module. No other file changes, and the
addon does not need rebuilding.
`ldd <module directory>/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64-0.35.5.node` shows which
`libvips-cpp.so.8.18.7` the addon resolves.
A module package's own `ekklesia.package.json` records the SHA-256 of each file it shipped, so a
modified tree is no longer the package Ekklesia verified at installation. Install it as your own
build.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,25 @@
#!/bin/sh
# Stands in for curl in sharp-libvips's build/posix.sh, so the build reads only this artifact's
# sources. Given the URL posix.sh would download, it prints the recorded local copy, or with -O
# writes it to the current directory under the URL's file name. A URL with no recorded copy fails:
# the build never reaches the network for source.
set -eu
write=""
url=""
for argument in "$@"; do
case "$argument" in
-O) write=yes ;;
-*) ;;
*) url="$argument" ;;
esac
done
file="$(awk -F '\t' -v url="$url" '$1 == url { print $2 }' /packaging/offline/url-map.tsv)"
if [ -z "$file" ]; then
echo "offline-curl: no recorded source for $url" >&2
exit 22
fi
if [ -n "$write" ]; then
cp "/packaging/offline/files/$file" "./${url##*/}"
else
cat "/packaging/offline/files/$file"
fi
@@ -0,0 +1,61 @@
#!/bin/sh
# Prepares an extracted sharp-libvips v1.3.4 source tree to build lib/libvips-cpp.so.8.18.7 from this
# artifact's sources alone. BUILDING.md describes each step.
#
# sh build/prepare-offline.sh <extracted sharp-libvips-1.3.4 directory>
#
# Only the source acquisition changes; nothing that is compiled does:
# - every archive and patch build/posix.sh downloads is read from this artifact (offline-curl.sh);
# - librsvg's Rust crates come from this artifact as a vendored directory source, and its lockfile
# is the resolved one recorded here instead of a fresh `cargo update` against crates.io;
# - platforms/linux-x64/Dockerfile installs the exact nightly Rust toolchain the binary records.
set -eu
here="$(cd "$(dirname "$0")/.." && pwd)"
tree="$(cd "$1" && pwd)"
[ -f "$tree/build/posix.sh" ] && [ -f "$tree/platforms/linux-x64/Dockerfile" ] || {
echo "prepare-offline: $1 is not an extracted sharp-libvips source tree." >&2
exit 1
}
offline="$tree/offline"
rm -rf "$offline"
mkdir -p "$offline/files" "$offline/vendor"
cp "$here"/sources/*.tar.* "$here"/patches/*.patch "$offline/files/"
cp "$tree/THIRD-PARTY-NOTICES.md" "$offline/files/THIRD-PARTY-NOTICES.md"
cp "$here/build/url-map.tsv" "$here/build/Cargo.lock" "$offline/"
cp "$here/build/offline-curl.sh" "$offline/offline-curl.sh"
chmod 755 "$offline/offline-curl.sh"
for crate in "$here"/sources/crates/*.crate; do
tar -xzf "$crate" -C "$offline/vendor"
name="$(basename "$crate" .crate)"
checksum="$(sha256sum "$crate" | cut -d ' ' -f 1)"
printf '{"files":{},"package":"%s"}\n' "$checksum" > "$offline/vendor/$name/.cargo-checksum.json"
done
cat > "$offline/cargo-config.toml" <<'CONFIG'
[source.crates-io]
replace-with = "vendored-sources"
[source.vendored-sources]
directory = "/packaging/offline/vendor"
[net]
offline = true
CONFIG
posix="$tree/build/posix.sh"
grep -q '^CURL="curl ' "$posix" && grep -q '^cargo update --workspace$' "$posix" || {
echo "prepare-offline: build/posix.sh is not the v1.3.4 script this artifact was prepared for." >&2
exit 1
}
sed -i \
-e 's|^CURL="curl .*"$|CURL="/packaging/offline/offline-curl.sh"\nexport CARGO_NET_OFFLINE=true\nmkdir -p "${CARGO_HOME:-$HOME/.cargo}" \&\& cp /packaging/offline/cargo-config.toml "${CARGO_HOME:-$HOME/.cargo}/config.toml"|' \
-e 's|^cargo update --workspace$|cp /packaging/offline/Cargo.lock Cargo.lock|' \
"$posix"
dockerfile="$tree/platforms/linux-x64/Dockerfile"
grep -q -- '--default-toolchain nightly ' "$dockerfile" || {
echo "prepare-offline: platforms/linux-x64/Dockerfile is not the v1.3.4 one." >&2
exit 1
}
sed -i 's|--default-toolchain nightly |--default-toolchain nightly-2026-09-27 |' "$dockerfile"
echo "prepare-offline: $tree builds from this artifact's sources."
@@ -0,0 +1,33 @@
https://cairographics.org/releases/cairo-1.18.6.tar.xz cairo-1.18.6.tar.xz
https://cairographics.org/releases/pixman-0.46.4.tar.gz pixman-0.46.4.tar.gz
https://download.gnome.org/sources/glib/2.90/glib-2.90.0.tar.xz glib-2.90.0.tar.xz
https://download.gnome.org/sources/librsvg/2.63/librsvg-2.63.2.tar.xz librsvg-2.63.2.tar.xz
https://download.gnome.org/sources/libxml2/2.15/libxml2-2.15.4.tar.xz libxml2-2.15.4.tar.xz
https://download.gnome.org/sources/pango/1.58/pango-1.58.2.tar.xz pango-1.58.2.tar.xz
https://gist.github.com/kleisauke/284d685efa00908da99ea6afbaaf39ae/raw/bdad5489a61c217850631571caf57f5db6ea8b2c/glib-without-gregex.patch glib-without-gregex.patch
https://gist.githubusercontent.com/lovell/313a6901e9db1bf285f2a1f1180499e4/raw/3988223c7dfa4d22745d9392034b0117abef1446/libvips-cpp-soversion.patch libvips-cpp-soversion.patch
https://github.com/GNOME/librsvg/commit/9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch librsvg-9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch
https://github.com/dloebl/cgif/archive/v0.5.4.tar.gz cgif-0.5.4.tar.gz
https://github.com/freetype/freetype/archive/VER-2-14-3.tar.gz freetype-2.14.3.tar.gz
https://github.com/fribidi/fribidi/releases/download/v1.0.17/fribidi-1.0.17.tar.xz fribidi-1.0.17.tar.xz
https://github.com/google/highway/archive/1.4.0.tar.gz highway-1.4.0.tar.gz
https://github.com/google/libultrahdr/archive/v2.0.2.tar.gz libultrahdr-2.0.2.tar.gz
https://github.com/harfbuzz/harfbuzz/archive/14.5.0.tar.gz harfbuzz-14.5.0.tar.gz
https://github.com/libarchive/libarchive/releases/download/v3.8.9/libarchive-3.8.9.tar.xz libarchive-3.8.9.tar.xz
https://github.com/libexif/libexif/releases/download/v0.6.26/libexif-0.6.26.tar.xz libexif-0.6.26.tar.xz
https://github.com/libexpat/libexpat/releases/download/R_2_8_5/expat-2.8.5.tar.xz expat-2.8.5.tar.xz
https://github.com/libffi/libffi/releases/download/v3.8.0/libffi-3.8.0.tar.gz libffi-3.8.0.tar.gz
https://github.com/libvips/libvips/releases/download/v8.18.7/vips-8.18.7.tar.xz vips-8.18.7.tar.xz
https://github.com/lovell/libimagequant/archive/v2.4.1.tar.gz libimagequant-2.4.1.tar.gz
https://github.com/mm2/Little-CMS/releases/download/lcms2.19.1/lcms2-2.19.1.tar.gz lcms2-2.19.1.tar.gz
https://github.com/mozilla/mozjpeg/archive/0826579.tar.gz mozjpeg-0826579.tar.gz
https://github.com/mozilla/mozjpeg/commit/f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch mozjpeg-f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch
https://github.com/pnggroup/libpng/archive/v1.6.58.tar.gz libpng-1.6.58.tar.gz
https://github.com/strukturag/libheif/releases/download/v1.23.5/libheif-1.23.5.tar.gz libheif-1.23.5.tar.gz
https://github.com/zlib-ng/zlib-ng/archive/2.3.3.tar.gz zlib-ng-2.3.3.tar.gz
https://gitlab.com/libtiff/libtiff/-/archive/v4.7.2/libtiff-v4.7.2.tar.gz libtiff-v4.7.2.tar.gz
https://gitlab.freedesktop.org/fontconfig/fontconfig/-/archive/2.18.3/fontconfig-2.18.3.tar.gz fontconfig-2.18.3.tar.gz
https://patch-diff.githubusercontent.com/raw/google/libultrahdr/pull/383.patch libultrahdr-pull-383.patch
https://raw.githubusercontent.com/lovell/sharp-libvips/main/THIRD-PARTY-NOTICES.md THIRD-PARTY-NOTICES.md
https://storage.googleapis.com/aom-releases/libaom-3.15.1.tar.gz libaom-3.15.1.tar.gz
https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.6.0.tar.gz libwebp-1.6.0.tar.gz
1 https://cairographics.org/releases/cairo-1.18.6.tar.xz cairo-1.18.6.tar.xz
2 https://cairographics.org/releases/pixman-0.46.4.tar.gz pixman-0.46.4.tar.gz
3 https://download.gnome.org/sources/glib/2.90/glib-2.90.0.tar.xz glib-2.90.0.tar.xz
4 https://download.gnome.org/sources/librsvg/2.63/librsvg-2.63.2.tar.xz librsvg-2.63.2.tar.xz
5 https://download.gnome.org/sources/libxml2/2.15/libxml2-2.15.4.tar.xz libxml2-2.15.4.tar.xz
6 https://download.gnome.org/sources/pango/1.58/pango-1.58.2.tar.xz pango-1.58.2.tar.xz
7 https://gist.github.com/kleisauke/284d685efa00908da99ea6afbaaf39ae/raw/bdad5489a61c217850631571caf57f5db6ea8b2c/glib-without-gregex.patch glib-without-gregex.patch
8 https://gist.githubusercontent.com/lovell/313a6901e9db1bf285f2a1f1180499e4/raw/3988223c7dfa4d22745d9392034b0117abef1446/libvips-cpp-soversion.patch libvips-cpp-soversion.patch
9 https://github.com/GNOME/librsvg/commit/9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch librsvg-9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch
10 https://github.com/dloebl/cgif/archive/v0.5.4.tar.gz cgif-0.5.4.tar.gz
11 https://github.com/freetype/freetype/archive/VER-2-14-3.tar.gz freetype-2.14.3.tar.gz
12 https://github.com/fribidi/fribidi/releases/download/v1.0.17/fribidi-1.0.17.tar.xz fribidi-1.0.17.tar.xz
13 https://github.com/google/highway/archive/1.4.0.tar.gz highway-1.4.0.tar.gz
14 https://github.com/google/libultrahdr/archive/v2.0.2.tar.gz libultrahdr-2.0.2.tar.gz
15 https://github.com/harfbuzz/harfbuzz/archive/14.5.0.tar.gz harfbuzz-14.5.0.tar.gz
16 https://github.com/libarchive/libarchive/releases/download/v3.8.9/libarchive-3.8.9.tar.xz libarchive-3.8.9.tar.xz
17 https://github.com/libexif/libexif/releases/download/v0.6.26/libexif-0.6.26.tar.xz libexif-0.6.26.tar.xz
18 https://github.com/libexpat/libexpat/releases/download/R_2_8_5/expat-2.8.5.tar.xz expat-2.8.5.tar.xz
19 https://github.com/libffi/libffi/releases/download/v3.8.0/libffi-3.8.0.tar.gz libffi-3.8.0.tar.gz
20 https://github.com/libvips/libvips/releases/download/v8.18.7/vips-8.18.7.tar.xz vips-8.18.7.tar.xz
21 https://github.com/lovell/libimagequant/archive/v2.4.1.tar.gz libimagequant-2.4.1.tar.gz
22 https://github.com/mm2/Little-CMS/releases/download/lcms2.19.1/lcms2-2.19.1.tar.gz lcms2-2.19.1.tar.gz
23 https://github.com/mozilla/mozjpeg/archive/0826579.tar.gz mozjpeg-0826579.tar.gz
24 https://github.com/mozilla/mozjpeg/commit/f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch mozjpeg-f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch
25 https://github.com/pnggroup/libpng/archive/v1.6.58.tar.gz libpng-1.6.58.tar.gz
26 https://github.com/strukturag/libheif/releases/download/v1.23.5/libheif-1.23.5.tar.gz libheif-1.23.5.tar.gz
27 https://github.com/zlib-ng/zlib-ng/archive/2.3.3.tar.gz zlib-ng-2.3.3.tar.gz
28 https://gitlab.com/libtiff/libtiff/-/archive/v4.7.2/libtiff-v4.7.2.tar.gz libtiff-v4.7.2.tar.gz
29 https://gitlab.freedesktop.org/fontconfig/fontconfig/-/archive/2.18.3/fontconfig-2.18.3.tar.gz fontconfig-2.18.3.tar.gz
30 https://patch-diff.githubusercontent.com/raw/google/libultrahdr/pull/383.patch libultrahdr-pull-383.patch
31 https://raw.githubusercontent.com/lovell/sharp-libvips/main/THIRD-PARTY-NOTICES.md THIRD-PARTY-NOTICES.md
32 https://storage.googleapis.com/aom-releases/libaom-3.15.1.tar.gz libaom-3.15.1.tar.gz
33 https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.6.0.tar.gz libwebp-1.6.0.tar.gz
File diff suppressed because it is too large Load Diff
+3 -1
View File
@@ -1,15 +1,17 @@
{ {
"name": "@ekklesia/module-pack", "name": "@ekklesia/module-pack",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
"files": [ "files": [
"bin", "bin",
"corresponding-sources",
"dist", "dist",
"license-supplements", "license-supplements",
"licenses" "licenses"
], ],
"bin": { "bin": {
"ekklesia-corresponding-source": "./bin/ekklesia-corresponding-source.mjs",
"ekklesia-pack-module": "./bin/ekklesia-pack-module.mjs" "ekklesia-pack-module": "./bin/ekklesia-pack-module.mjs"
}, },
"exports": { "exports": {
@@ -0,0 +1,155 @@
#!/usr/bin/env node
import path from "node:path";
import process from "node:process";
import { pathToFileURL } from "node:url";
import {
acquireCorrespondingSources,
buildCorrespondingSource,
CorrespondingSourceError,
verifyRedistribution,
} from "./corresponding-source.js";
/**
* `ekklesia-corresponding-source` — corresponding-source artifacts for native packages (ADR 0055).
*
* ekklesia-corresponding-source acquire (--package <module package> | --recipe <name>@<version>) --cache <directory>
* Downloads every recorded source file the package's covered native dependencies (or the named
* recipe) need into the verified cache. The only subcommand that uses the network.
* ekklesia-corresponding-source build --package <module package> --cache <directory> --out <directory>
* Offline. Writes <package>-corresponding-source.tar.gz, its .sha256, <package>.redistribution.json,
* and <package>.source-notice.txt.
* ekklesia-corresponding-source verify --package <module package> [--source <archive> --metadata <json>]
* The public-distribution gate. Exits 0 only when the package is approved.
*/
const USAGE = [
"Usage:",
" ekklesia-corresponding-source acquire (--package <archive> | --recipe <name>@<version>) --cache <directory>",
" ekklesia-corresponding-source build --package <archive> --cache <directory> --out <directory>",
" ekklesia-corresponding-source verify --package <archive> [--source <archive> --metadata <file>]",
].join("\n");
const OPTIONS: Readonly<Record<string, readonly string[]>> = Object.freeze({
acquire: ["--package", "--recipe", "--cache"],
build: ["--package", "--cache", "--out"],
verify: ["--package", "--source", "--metadata"],
});
const REQUIRED: Readonly<Record<string, readonly string[]>> = Object.freeze({
acquire: ["--cache"],
build: ["--package", "--cache", "--out"],
verify: ["--package"],
});
class ArgumentsError extends Error {}
export function parseCorrespondingSourceArguments(argv: readonly string[]): {
readonly command: "acquire" | "build" | "verify";
readonly values: ReadonlyMap<string, string>;
} {
const [command, ...rest] = argv;
const allowed = command === undefined ? undefined : OPTIONS[command];
if (allowed === undefined) {
throw new ArgumentsError(USAGE);
}
const values = new Map<string, string>();
for (let index = 0; index < rest.length; index += 2) {
const option = rest[index] ?? "";
const value = rest[index + 1];
if (
!allowed.includes(option) ||
value === undefined ||
value.startsWith("--") ||
values.has(option)
) {
throw new ArgumentsError(USAGE);
}
values.set(option, option === "--recipe" ? value : path.resolve(value));
}
if (command === "acquire" && values.has("--package") === values.has("--recipe")) {
throw new ArgumentsError(USAGE);
}
if ((REQUIRED[command as string] ?? []).some((option) => !values.has(option))) {
throw new ArgumentsError(USAGE);
}
return { command: command as "acquire" | "build" | "verify", values };
}
async function main(): Promise<number> {
try {
const { command, values } = parseCorrespondingSourceArguments(process.argv.slice(2));
const packageArchivePath = values.get("--package") ?? "";
if (command === "acquire") {
const recipe = values.get("--recipe");
const at = recipe?.lastIndexOf("@") ?? -1;
if (recipe !== undefined && at <= 0) {
throw new ArgumentsError(USAGE);
}
const result = await acquireCorrespondingSources({
cacheDirectory: values.get("--cache") ?? "",
...(recipe === undefined
? { packageArchivePath }
: { recipes: [{ name: recipe.slice(0, at), version: recipe.slice(at + 1) }] }),
});
process.stdout.write(
`Source cache holds all ${result.files} recorded files (${result.bytes} bytes); fetched ${result.fetched}.\n`,
);
} else if (command === "build") {
const result = await buildCorrespondingSource({
cacheDirectory: values.get("--cache") ?? "",
outDirectory: values.get("--out") ?? "",
packageArchivePath,
});
if (!result.required) {
process.stdout.write(
"The package redistributes no covered native code: no artifact is needed.\n",
);
} else {
process.stdout.write(
`Wrote ${path.basename(result.sourceArchivePath ?? "")}, ${result.sourceBytes} bytes\nsha256 ${result.sourceSha256}\n`,
);
}
} else {
const metadataPath = values.get("--metadata");
const sourceArchivePath = values.get("--source");
const approval = await verifyRedistribution({
packageArchivePath,
...(metadataPath === undefined ? {} : { metadataPath }),
...(sourceArchivePath === undefined ? {} : { sourceArchivePath }),
});
process.stdout.write(
approval.correspondingSource === undefined
? `Approved: ${approval.package.file} redistributes no covered native code.\n`
: `Approved: ${approval.package.file} with ${approval.correspondingSource.file} (sha256 ${approval.correspondingSource.sha256}).\n`,
);
}
return 0;
} catch (error) {
if (error instanceof ArgumentsError) {
process.stderr.write(`${error.message}\n`);
return 64;
}
if (error instanceof CorrespondingSourceError) {
process.stderr.write(
`${error.code}: ${error.message}${error.subject ? ` (${error.subject})` : ""}\n`,
);
return 1;
}
process.stderr.write("ekklesia-corresponding-source failed unexpectedly.\n");
return 1;
}
}
function isDirectExecution(): boolean {
const entrypoint = process.argv[1];
// Run through the committed launcher, which imports this file, or directly.
return (
entrypoint !== undefined &&
(import.meta.url === pathToFileURL(path.resolve(entrypoint)).href ||
path.basename(entrypoint) === "ekklesia-corresponding-source.mjs" ||
path.basename(entrypoint) === "ekklesia-corresponding-source")
);
}
if (isDirectExecution()) {
process.exitCode = await main();
}
File diff suppressed because it is too large Load Diff
+1
View File
@@ -1 +1,2 @@
export * from "./corresponding-source.js";
export * from "./pack-module.js"; export * from "./pack-module.js";
+99 -1
View File
@@ -1,4 +1,4 @@
import { gzipSync } from "node:zlib"; import { gunzipSync, gzipSync } from "node:zlib";
/** /**
* The deterministic archive encoding of Ekklesia Module Package v1 (ADR 0049). * The deterministic archive encoding of Ekklesia Module Package v1 (ADR 0049).
@@ -107,3 +107,101 @@ export function deterministicTarGz(topLevel: string, files: ReadonlyMap<string,
parts.push(Buffer.alloc(BLOCK * 2)); parts.push(Buffer.alloc(BLOCK * 2));
return gzipSync(Buffer.concat(parts), { level: GZIP_LEVEL }); return gzipSync(Buffer.concat(parts), { level: GZIP_LEVEL });
} }
/** One entry of a ustar archive, as read back. */
export interface TarEntry {
readonly path: string;
readonly directory: boolean;
readonly data: Buffer;
}
export class TarFormatError extends Error {
constructor() {
super("An archive is not in Ekklesia's deterministic ustar encoding.");
this.name = "TarFormatError";
}
}
function field(block: Buffer, offset: number, length: number): string {
const raw = block.subarray(offset, offset + length);
const end = raw.indexOf(0);
return raw.subarray(0, end === -1 ? length : end).toString("latin1");
}
/**
* Reads an archive written by `deterministicTarGz` back into its entries, refusing anything that
* encoding never produces: a link or special file, another header type, a bad checksum, or data
* after the end blocks. Whether the archive is byte-for-byte canonical is checked separately, by
* writing the entries again (`isCanonicalTarGz`).
*/
export function readTarGz(archive: Buffer): readonly TarEntry[] {
let raw: Buffer;
try {
raw = gunzipSync(archive);
} catch {
throw new TarFormatError();
}
const entries: TarEntry[] = [];
let offset = 0;
for (;;) {
if (offset + BLOCK > raw.length) {
throw new TarFormatError();
}
const block = raw.subarray(offset, offset + BLOCK);
if (block.every((byte) => byte === 0)) {
break;
}
const stored = Number.parseInt(field(block, 148, 8).trim() || "x", 8);
let checksum = 0;
for (let index = 0; index < BLOCK; index += 1) {
checksum += index >= 148 && index < 156 ? 32 : (block[index] ?? 0);
}
const typeflag = field(block, 156, 1);
const size = Number.parseInt(field(block, 124, 12) || "x", 8);
if (
stored !== checksum ||
(typeflag !== "0" && typeflag !== "5") ||
!Number.isSafeInteger(size)
) {
throw new TarFormatError();
}
const name = field(block, 0, 100);
const prefix = field(block, 345, 155);
const joined = prefix === "" ? name : `${prefix}/${name}`;
const directory = typeflag === "5";
const start = offset + BLOCK;
if (start + size > raw.length) {
throw new TarFormatError();
}
entries.push({
data: Buffer.from(raw.subarray(start, start + size)),
directory,
path: directory ? joined.replace(/\/$/u, "") : joined,
});
offset = start + Math.ceil(size / BLOCK) * BLOCK;
}
return entries;
}
/** Whether `archive` is exactly what `deterministicTarGz` writes for its own entries: the same
* order, headers, and padding. Compared uncompressed, so a different zlib cannot matter. */
export function isCanonicalTarGz(archive: Buffer, entries: readonly TarEntry[]): boolean {
const topLevel = entries[0];
if (topLevel === undefined || !topLevel.directory || topLevel.path.includes("/")) {
return false;
}
const files = new Map<string, Buffer>();
for (const entry of entries) {
if (!entry.directory) {
if (!entry.path.startsWith(`${topLevel.path}/`)) {
return false;
}
files.set(entry.path.slice(topLevel.path.length + 1), entry.data);
}
}
try {
return gunzipSync(deterministicTarGz(topLevel.path, files)).equals(gunzipSync(archive));
} catch {
return false;
}
}
@@ -0,0 +1,342 @@
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { existsSync } from "node:fs";
import { readFile, realpath, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { currentModulePackageHost, verifyModulePackage } from "@ekklesia/core/modules";
import { afterEach, describe, expect, it } from "vitest";
import {
buildCorrespondingSource,
CorrespondingSourceError,
loadSourceRecipes,
verifyRedistribution,
} from "../src/corresponding-source.js";
import { packModule } from "../src/index.js";
import { NATIVE_TARGETS, nativeClosure } from "../src/native.js";
import {
contentDigest,
DEFAULT_SUPPLEMENTS_DIRECTORY,
loadLicenseSupplements,
} from "../src/supplements.js";
import { readTarGz } from "../src/tar.js";
import {
cleanupFixtures,
installedPackageFile,
nativeFixtureModule,
SOURCE_REVISION,
scratchDirectory,
} from "./support/fixture-module.js";
/**
* ADR 0055's first complete proof: the native proof module with sharp 0.35.5, whose libvips 1.3.4
* binary contains LGPL and MPL components, and its real corresponding-source artifact.
*
* The recipe checks run everywhere. Building the artifact needs the recorded sources in a verified
* cache, which `pnpm test:corresponding-source` acquires once and passes in as
* EKKLESIA_CORRESPONDING_SOURCE_CACHE; without it, that part is skipped rather than downloading
* about 200 MB on every run.
*/
const execFileAsync = promisify(execFile);
const packageRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
const repositoryRoot = path.resolve(packageRoot, "..", "..");
const LIBVIPS = "@img/sharp-libvips-linux-x64";
const cache = process.env.EKKLESIA_CORRESPONDING_SOURCE_CACHE;
const host = currentModulePackageHost();
const onTargetHost =
host.platform === "linux" && host.arch === "x64" && host.libc?.family === "glibc";
afterEach(cleanupFixtures);
// `pnpm test:corresponding-source` asks for the full proof: then a missing cache is a failure, not a
// skip.
it.runIf(process.env.EKKLESIA_CORRESPONDING_SOURCE_REQUIRED === "1")(
"has the verified source cache it was asked to use, on a linux-x64-glibc host",
() => {
expect(onTargetHost).toBe(true);
expect(cache !== undefined && existsSync(cache)).toBe(true);
},
);
describe("the libvips 1.3.4 source recipe", () => {
it("loads intact, matches its reviewed supplement exactly, and pins the installed package", async () => {
const recipes = await loadSourceRecipes();
const [loaded] = recipes.get(LIBVIPS) ?? [];
expect(loaded?.recipe.package).toMatchObject({
license: "LGPL-3.0-or-later",
target: "linux-x64-glibc",
version: "1.3.4",
});
// Prepared against exactly the reviewed supplement for these package bytes.
const supplementJson = await readFile(
path.join(DEFAULT_SUPPLEMENTS_DIRECTORY, LIBVIPS, "1.3.4", "supplement.json"),
);
expect(loaded?.recipe.supplement.sha256).toBe(
createHash("sha256").update(supplementJson).digest("hex"),
);
const supplement = (await loadLicenseSupplements())
.get(LIBVIPS)
?.find((candidate) => candidate.version === "1.3.4");
expect(loaded?.recipe.package.contentDigest).toBe(supplement?.contentDigest);
// And those are the bytes sharp 0.35.5 installs.
const root = await realpath(await installedPackageFile("sharp"));
const target = NATIVE_TARGETS.get("linux-x64-glibc");
if (target === undefined) throw new Error("target");
const installed = (await nativeClosure([{ name: "sharp", root }], target)).find(
(entry) => entry.name === LIBVIPS,
);
expect(installed && contentDigest(installed.files)).toBe(loaded?.recipe.package.contentDigest);
// Every component the supplement records inside the binary maps to a source component. The one
// exception is the package's own entry, which carries the standard texts of its declared licence.
const referenced = new Set(
(loaded?.recipe.components ?? []).flatMap((component) =>
component.licenseMaterial.map((material) => `${material.name} ${material.version}`),
),
);
const unmapped = (supplement?.components ?? [])
.map((component) => `${component.name} ${component.version}`)
.filter((component) => !referenced.has(component));
expect(unmapped).toEqual([`${LIBVIPS} 1.3.4`]);
const components = loaded?.recipe.components ?? [];
const byRole = (role: string) =>
components.filter((component) => component.role === role).length;
expect({
buildDefinition: byRole("build-definition"),
buildInput: byRole("build-input"),
compiled: byRole("compiled"),
}).toEqual({
buildDefinition: 1,
buildInput: 206,
compiled: 176,
});
// The five patches build/posix.sh applies, and the Rust standard library of the recorded nightly.
expect(
components.flatMap((component) => component.patches.map((patch) => patch.file)).sort(),
).toEqual([
"patches/glib-without-gregex.patch",
"patches/librsvg-9106011db93d701728ac2c9da50c9ab2c1bb5dc6.patch",
"patches/libultrahdr-pull-383.patch",
"patches/libvips-cpp-soversion.patch",
"patches/mozjpeg-f90668e0e4fb79c81e1f24a0ccc0e2090af761bf.patch",
]);
expect(
components.find((component) => component.name === "Rust standard library")?.sources[0]?.file,
).toBe("sources/rust/rust-src-nightly-2026-09-27.tar.xz");
// gvdb ships inside glib's archive; proxy-libintl is never built for this target.
expect(components.find((component) => component.name === "gvdb")?.includedIn).toEqual({
component: "glib",
path: "subprojects/gvdb",
});
expect(components.some((component) => component.name === "proxy-libintl")).toBe(false);
expect(loaded?.recipe.exclusions.map((exclusion) => exclusion.name)).toContain("proxy-libintl");
});
});
async function packNative(): Promise<string> {
const moduleDirectory = await nativeFixtureModule();
const outDirectory = await scratchDirectory("ekklesia-native-out-");
const result = await packModule({
licenseFile: path.join(repositoryRoot, "LICENSE"),
moduleDirectory,
nodeRange: ">=24.11.0 <25",
outDirectory,
sourceRevision: SOURCE_REVISION,
target: "linux-x64-glibc",
});
return result.archivePath;
}
describe.runIf(onTargetHost && cache !== undefined && existsSync(cache ?? ""))(
"the native proof's corresponding source, from the verified cache",
() => {
it("builds deterministically, is approved by the gate, and fails closed without it", {
timeout: 600_000,
}, async () => {
const first = await packNative();
const second = await packNative();
expect((await readFile(first)).equals(await readFile(second))).toBe(true);
const outA = await scratchDirectory("ekklesia-source-a-");
const outB = path.join(await scratchDirectory("ekklesia-unrelated-"), "nested", "out");
const a = await buildCorrespondingSource({
cacheDirectory: cache ?? "",
outDirectory: outA,
packageArchivePath: first,
});
const b = await buildCorrespondingSource({
cacheDirectory: cache ?? "",
outDirectory: outB,
packageArchivePath: second,
});
expect(a.required).toBe(true);
expect(a.sourceSha256).toBe(b.sourceSha256);
expect(
(await readFile(a.sourceArchivePath ?? "")).equals(
await readFile(b.sourceArchivePath ?? ""),
),
).toBe(true);
expect(path.basename(a.sourceArchivePath ?? "")).toBe(
"native-proof-0.1.0-linux-x64-glibc-corresponding-source.tar.gz",
);
// What it holds: every recorded source, as bytes, and nothing unrecorded.
const archive = await readFile(a.sourceArchivePath ?? "");
const entries = readTarGz(archive);
const top = "native-proof-0.1.0-linux-x64-glibc-corresponding-source";
const files = entries
.filter((entry) => !entry.directory)
.map((entry) => entry.path.slice(top.length + 1));
const manifest = JSON.parse(
entries
.find((entry) => entry.path === `${top}/corresponding-source.json`)
?.data.toString("utf8") ?? "{}",
);
const dependency = manifest.dependencies[0];
expect(dependency).toMatchObject({
name: LIBVIPS,
version: "1.3.4",
binaries: ["lib/libvips-cpp.so.8.18.7"],
});
const base = `dependencies/${LIBVIPS}/1.3.4`;
const sources = files.filter((file) => file.startsWith(`${base}/sources/`));
expect(sources.filter((file) => file.startsWith(`${base}/sources/crates/`))).toHaveLength(
353,
);
expect(
sources.filter((file) => /\/sources\/[^/]+\.tar\.(?:gz|xz)$/u.test(file)),
).toHaveLength(28);
expect(files.filter((file) => file.startsWith(`${base}/patches/`))).toHaveLength(5);
for (const required of [
"BUILDING.md",
"PROVENANCE.md",
"recipe.json",
"licenses/supplement.json",
"build/prepare-offline.sh",
"build/Cargo.lock",
"sources/rust/rust-src-nightly-2026-09-27.tar.xz",
]) {
expect(files, required).toContain(`${base}/${required}`);
}
expect(new Set(manifest.files.map((file: { path: string }) => file.path))).toEqual(
new Set(files.filter((file) => file !== "corresponding-source.json")),
);
// The binary-source relationship, by digest.
const metadata = JSON.parse(await readFile(a.metadataPath ?? "", "utf8"));
const packageSha256 = createHash("sha256")
.update(await readFile(first))
.digest("hex");
expect(metadata).toMatchObject({
correspondingSource: {
file: path.basename(a.sourceArchivePath ?? ""),
sha256: a.sourceSha256,
size: archive.length,
},
covered: [
{
contentDigest:
"ekklesia-content-v1:sha256:2927fe11493461fa34847191d0b9b1c6441007c3e99f306e5708cb07fb0a5e59",
name: LIBVIPS,
version: "1.3.4",
},
],
package: {
file: path.basename(first),
module: { id: "native-proof", version: "0.1.0" },
sha256: packageSha256,
target: "linux-x64-glibc",
},
});
expect(metadata.notice).toContain("GNU LGPL and MPL");
expect(metadata.notice).toContain(`SHA-256 ${a.sourceSha256}`);
// The gate approves exactly this, and fails closed when it is removed or corrupted.
await expect(
verifyRedistribution({
metadataPath: a.metadataPath ?? "",
packageArchivePath: first,
sourceArchivePath: a.sourceArchivePath ?? "",
}),
).resolves.toMatchObject({ approved: true, correspondingSource: { sha256: a.sourceSha256 } });
const missing = verifyRedistribution({
metadataPath: a.metadataPath ?? "",
packageArchivePath: first,
});
await expect(missing).rejects.toBeInstanceOf(CorrespondingSourceError);
await expect(missing).rejects.toMatchObject({
code: "CORRESPONDING_SOURCE_ARTIFACT_MISSING",
});
const corrupted = Buffer.from(archive);
const flip = Math.floor(corrupted.length / 3);
corrupted[flip] = (corrupted[flip] ?? 0) ^ 0x01;
await writeFile(b.sourceArchivePath ?? "", corrupted);
await expect(
verifyRedistribution({
metadataPath: b.metadataPath ?? "",
packageArchivePath: second,
sourceArchivePath: b.sourceArchivePath ?? "",
}),
).rejects.toMatchObject({
code: expect.stringMatching(
/^CORRESPONDING_SOURCE_(?:ARTIFACT_INVALID|MANIFEST_MISMATCH|METADATA_MISMATCH)$/u,
),
});
await rm(b.sourceArchivePath ?? "");
await expect(
verifyRedistribution({
metadataPath: b.metadataPath ?? "",
packageArchivePath: second,
sourceArchivePath: b.sourceArchivePath ?? "",
}),
).rejects.toMatchObject({ code: "CORRESPONDING_SOURCE_ARTIFACT_MISSING" });
});
it("leaves the binary package itself unchanged: it verifies, and sharp runs from it", {
timeout: 300_000,
}, async () => {
const archivePath = await packNative();
const archive = await readFile(archivePath);
const extractTo = path.join(
await scratchDirectory("ekklesia-native-installed-"),
"native-proof",
);
await expect(
verifyModulePackage({
archivePath,
expectedSha256: createHash("sha256").update(archive).digest("hex"),
extractTo,
platform: {
databaseRuntime: { "drizzle-orm": "0.45.2" },
moduleSdkVersion: "0.2.1",
platformVersion: "0.2.1",
},
}),
).resolves.toMatchObject({ moduleId: "native-proof", target: "linux-x64-glibc" });
// The source archive never travels inside the package.
expect(readTarGz(archive).some((entry) => /corresponding-source/u.test(entry.path))).toBe(
false,
);
const { stdout } = await execFileAsync(
process.execPath,
[
"--input-type=module",
"-e",
`const m = await import(${JSON.stringify(`file://${extractTo}/server/index.js`)}); process.stdout.write(JSON.stringify(await m.proof()));`,
],
{ cwd: tmpdir(), env: {} },
);
expect(JSON.parse(stdout)).toEqual({
pixels: [255, 0, 0, 255, 0, 0, 255, 0, 0, 255, 0, 0],
png: { format: "png", height: 3, width: 5 },
resized: { channels: 3, height: 2, width: 2 },
});
});
},
);
@@ -0,0 +1,774 @@
import { createHash } from "node:crypto";
import { mkdir, readFile, rm, writeFile } from "node:fs/promises";
import path from "node:path";
import { gunzipSync, gzipSync } from "node:zlib";
import { afterEach, describe, expect, it } from "vitest";
import {
acquireCorrespondingSources,
buildCorrespondingSource,
CorrespondingSourceError,
type CorrespondingSourceErrorCode,
hasSourceObligations,
loadSourceRecipes,
verifyRedistribution,
} from "../src/corresponding-source.js";
import { contentDigest } from "../src/supplements.js";
import { deterministicTarGz, readTarGz } from "../src/tar.js";
import { cleanupFixtures, scratchDirectory } from "./support/fixture-module.js";
/**
* Corresponding-source artifacts and the public-distribution gate (ADR 0055), on small synthetic
* inputs: a fake LGPL native package, its reviewed supplement, its source recipe, and a source
* cache. The real libvips proof is `corresponding-source-native.test.ts`.
*/
afterEach(cleanupFixtures);
function sha256(data: string | Uint8Array): string {
return createHash("sha256").update(data).digest("hex");
}
const PACKAGE_FILE = "example-0.1.0-linux-x64-glibc.tar.gz";
const SOURCE_FILE = "example-0.1.0-linux-x64-glibc-corresponding-source.tar.gz";
const METADATA_FILE = "example-0.1.0-linux-x64-glibc.redistribution.json";
const TOP = "example-0.1.0-linux-x64-glibc-corresponding-source";
const SOURCES: Readonly<Record<string, string>> = {
"sources/fake-build-1.0.tar.gz": "build definitions for libfake\n",
"sources/fakelib-2.0.tar.gz": "fakelib 2.0 source\n",
"patches/fakelib-fix.patch": "--- a/x\n+++ b/x\n@@ -1 +1 @@\n-a\n+b\n",
"sources/helperlib-1.1.tar.gz": "helperlib 1.1 source\n",
"sources/codegen-0.3.crate": "codegen 0.3 build-only source\n",
};
interface FixtureOptions {
readonly version?: string;
readonly libraryBytes?: string;
readonly license?: string;
readonly notices?: string;
readonly recipe?: (recipe: Record<string, unknown>) => void;
readonly withRecipe?: boolean;
}
interface Fixture {
readonly root: string;
readonly packagePath: string;
readonly cache: string;
readonly recipesDirectory: string;
readonly supplementsDirectory: string;
}
function noticeEntry(name: string, version: string, licence: string, uses: string): string {
const rule = "-".repeat(80);
return `${rule}\n${name} ${version}\nLicense: ${licence}\nUsed in: ${uses}\n${rule}\n\n[LICENSE]\n\ntext\n`;
}
/** Writes a fake native package into a Module Package archive, its supplement, recipe, and cache. */
async function fixture(options: FixtureOptions = {}): Promise<Fixture> {
const version = options.version ?? "1.0.0";
const license = options.license ?? "LGPL-2.1-or-later";
const root = await scratchDirectory("ekklesia-corresponding-");
const nativeFiles = new Map<string, Buffer>([
["lib/libfake.so.1", Buffer.from(options.libraryBytes ?? `\u007fELF fake library ${version}`)],
["package.json", Buffer.from(JSON.stringify({ license, name: "fake-native", version }))],
]);
const digest = contentDigest(nativeFiles);
const packageFiles = new Map<string, Buffer>([
[
"ekklesia.package.json",
Buffer.from(
JSON.stringify({
module: { id: "example", version: "0.1.0" },
packageFormat: 1,
target: "linux-x64-glibc",
}),
),
],
[
"THIRD_PARTY_NOTICES",
Buffer.from(options.notices ?? noticeEntry("fake-native", version, license, "node_modules")),
],
...[...nativeFiles].map(([file, data]) => [`node_modules/fake-native/${file}`, data] as const),
]);
const packageDirectory = path.join(root, "package");
await mkdir(packageDirectory);
const packagePath = path.join(packageDirectory, PACKAGE_FILE);
await writeFile(packagePath, deterministicTarGz("example", packageFiles));
// The reviewed supplement for exactly these package bytes.
const supplementsDirectory = path.join(root, "supplements");
const supplementDirectory = path.join(supplementsDirectory, "fake-native", version);
await mkdir(path.join(supplementDirectory, "texts"), { recursive: true });
const material = async (content: string) => {
await writeFile(path.join(supplementDirectory, "texts", `${sha256(content)}.txt`), content);
return `texts/${sha256(content)}.txt`;
};
const supplement = {
supplementFormat: 1,
package: { name: "fake-native", version, license, contentDigest: digest },
statement: "Covers the libraries built into lib/libfake.so.1.",
components: [
{
name: "fakelib",
version: "2.0",
license: "LGPL-2.1-or-later",
source: "https://example.org/fakelib-2.0.tar.gz",
material: [
{ title: "COPYING", text: await material("GNU LESSER GENERAL PUBLIC LICENSE (fake)\n") },
],
},
{
name: "helperlib",
version: "1.1",
license: "MIT",
source: "https://example.org/helperlib-1.1.tar.gz",
material: [{ title: "LICENSE", text: await material("MIT License (fake)\n") }],
},
],
};
const supplementJson = `${JSON.stringify(supplement, null, 2)}\n`;
await writeFile(path.join(supplementDirectory, "supplement.json"), supplementJson);
await writeFile(path.join(supplementDirectory, "PROVENANCE.md"), "# How this was assembled\n");
// The recipe, and a cache holding the recorded bytes.
const recipesDirectory = path.join(root, "recipes");
const cache = path.join(root, "cache");
await mkdir(path.join(cache, "sha256"), { recursive: true });
const file = async (relative: string) => {
const content = SOURCES[relative] ?? "";
await writeFile(path.join(cache, "sha256", sha256(content)), content);
return {
file: relative,
origin: `https://example.org/${path.basename(relative)}`,
sha256: sha256(content),
size: Buffer.byteLength(content),
};
};
const building = "# Rebuilding libfake\n\nRun `make` in fake-build.\n";
const recipe: Record<string, unknown> = {
recipeFormat: 1,
package: {
name: "fake-native",
version,
license,
contentDigest: digest,
target: "linux-x64-glibc",
binaries: ["lib/libfake.so.1"],
},
supplement: { version, sha256: sha256(supplementJson) },
documents: [
{ path: "BUILDING.md", sha256: sha256(building), size: Buffer.byteLength(building) },
],
components: [
{
name: "fake-build",
version: "1.0",
role: "build-definition",
license: "MIT",
relationship: "The build definitions.",
licenseMaterial: [],
sources: [await file("sources/fake-build-1.0.tar.gz")],
patches: [],
},
{
name: "fakelib",
version: "2.0",
role: "compiled",
license: "LGPL-2.1-or-later",
relationship: "Compiled into lib/libfake.so.1.",
licenseMaterial: [{ name: "fakelib", version: "2.0" }],
sources: [await file("sources/fakelib-2.0.tar.gz")],
patches: [await file("patches/fakelib-fix.patch")],
},
{
name: "helperlib",
version: "1.1",
role: "compiled",
license: "MIT",
relationship: "Compiled into lib/libfake.so.1.",
licenseMaterial: [{ name: "helperlib", version: "1.1" }],
sources: [await file("sources/helperlib-1.1.tar.gz")],
patches: [],
},
{
name: "codegen",
version: "0.3",
role: "build-input",
license: "MIT",
relationship: "Needed only to build.",
licenseMaterial: [],
sources: [await file("sources/codegen-0.3.crate")],
patches: [],
},
],
exclusions: [{ name: "testlib", reason: "Used only by tests." }],
};
options.recipe?.(recipe);
if (options.withRecipe !== false) {
const recipeDirectory = path.join(recipesDirectory, "fake-native", version);
await mkdir(recipeDirectory, { recursive: true });
await writeFile(
path.join(recipeDirectory, "recipe.json"),
`${JSON.stringify(recipe, null, 2)}\n`,
);
await writeFile(path.join(recipeDirectory, "BUILDING.md"), building);
}
return { cache, packagePath, recipesDirectory, root, supplementsDirectory };
}
async function build(fx: Fixture, out = path.join(fx.root, "out")) {
return buildCorrespondingSource({
cacheDirectory: fx.cache,
outDirectory: out,
packageArchivePath: fx.packagePath,
recipesDirectory: fx.recipesDirectory,
supplementsDirectory: fx.supplementsDirectory,
});
}
function verify(fx: Fixture, source?: string, metadata?: string) {
return verifyRedistribution({
packageArchivePath: fx.packagePath,
recipesDirectory: fx.recipesDirectory,
supplementsDirectory: fx.supplementsDirectory,
...(source === undefined ? {} : { sourceArchivePath: source }),
...(metadata === undefined ? {} : { metadataPath: metadata }),
});
}
async function rejects(promise: Promise<unknown>, code: CorrespondingSourceErrorCode) {
const error = await promise.then(
() => undefined,
(caught: unknown) => caught,
);
expect(error).toBeInstanceOf(CorrespondingSourceError);
expect((error as CorrespondingSourceError).code).toBe(code);
return error as CorrespondingSourceError;
}
/** A built release, ready to tamper with: its source archive's files, and a way to write a changed
* archive (and metadata naming it) where the gate will look. */
async function release() {
const fx = await fixture();
const out = path.join(fx.root, "out");
await build(fx, out);
const sourcePath = path.join(out, SOURCE_FILE);
const metadataPath = path.join(out, METADATA_FILE);
const files = new Map<string, Buffer>();
for (const entry of readTarGz(await readFile(sourcePath))) {
if (!entry.directory) files.set(entry.path.slice(TOP.length + 1), entry.data);
}
/** Writes `changed` as the archive and keeps the metadata consistent with its bytes, so only the
* change itself can fail the gate. */
const write = async (changed: Map<string, Buffer>, archive?: Buffer) => {
const bytes = archive ?? deterministicTarGz(TOP, changed);
await writeFile(sourcePath, bytes);
const metadata = JSON.parse(await readFile(metadataPath, "utf8"));
const digest = sha256(bytes);
metadata.correspondingSource.sha256 = digest;
metadata.correspondingSource.size = bytes.length;
metadata.correspondingSource.manifestSha256 = sha256(
changed.get("corresponding-source.json") ?? "",
);
metadata.notice = metadata.notice.replace(/SHA-256 [0-9a-f]{64}/u, `SHA-256 ${digest}`);
await writeFile(metadataPath, `${JSON.stringify(metadata, null, 2)}\n`);
};
/** Rewrites the manifest's file list to match `changed`, as a careful forger would. */
const relist = (changed: Map<string, Buffer>) => {
const manifest = JSON.parse(changed.get("corresponding-source.json")?.toString("utf8") ?? "{}");
manifest.files = [...changed]
.filter(([file]) => file !== "corresponding-source.json")
.map(([file, data]) => ({ path: file, size: data.length, sha256: sha256(data) }))
.sort((left: { path: string }, right: { path: string }) => (left.path < right.path ? -1 : 1));
changed.set("corresponding-source.json", Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`));
return manifest;
};
return { files, fx, metadataPath, out, relist, sourcePath, write };
}
describe("building the corresponding-source artifact", () => {
it("writes a deterministic archive, its digest, the redistribution metadata, and the release notice", async () => {
const fx = await fixture();
const first = await build(fx, path.join(fx.root, "out-a"));
// A second build into an unrelated directory, from a moved cache and package copy.
const second = await build(
fx,
path.join(await scratchDirectory("ekklesia-elsewhere-"), "out-b"),
);
expect(first.required).toBe(true);
expect(first.sourceSha256).toBe(second.sourceSha256);
expect(await readFile(first.sourceArchivePath ?? "")).toEqual(
await readFile(second.sourceArchivePath ?? ""),
);
expect(await readFile(`${first.sourceArchivePath}.sha256`, "utf8")).toBe(
`${first.sourceSha256} ${SOURCE_FILE}\n`,
);
const entries = readTarGz(await readFile(first.sourceArchivePath ?? ""));
const files = entries
.filter((entry) => !entry.directory)
.map((entry) => entry.path.slice(TOP.length + 1));
expect(files).toEqual([
"README.md",
"SHA256SUMS",
"corresponding-source.json",
"dependencies/fake-native/1.0.0/BUILDING.md",
"dependencies/fake-native/1.0.0/PROVENANCE.md",
"dependencies/fake-native/1.0.0/licenses/supplement.json",
expect.stringMatching(
/^dependencies\/fake-native\/1\.0\.0\/licenses\/texts\/[0-9a-f]{64}\.txt$/u,
),
expect.stringMatching(
/^dependencies\/fake-native\/1\.0\.0\/licenses\/texts\/[0-9a-f]{64}\.txt$/u,
),
"dependencies/fake-native/1.0.0/patches/fakelib-fix.patch",
"dependencies/fake-native/1.0.0/recipe.json",
"dependencies/fake-native/1.0.0/sources/codegen-0.3.crate",
"dependencies/fake-native/1.0.0/sources/fake-build-1.0.tar.gz",
"dependencies/fake-native/1.0.0/sources/fakelib-2.0.tar.gz",
"dependencies/fake-native/1.0.0/sources/helperlib-1.1.tar.gz",
]);
// The actual bytes, not references to them.
const patch = entries.find((entry) => entry.path.endsWith("patches/fakelib-fix.patch"));
expect(patch?.data.toString("utf8")).toBe(SOURCES["patches/fakelib-fix.patch"]);
const manifest = JSON.parse(
entries
.find((entry) => entry.path.endsWith("/corresponding-source.json"))
?.data.toString("utf8") ?? "{}",
);
expect(manifest.module).toEqual({ id: "example", target: "linux-x64-glibc", version: "0.1.0" });
expect(manifest.dependencies[0]).toMatchObject({
binaries: ["lib/libfake.so.1"],
exclusions: [{ name: "testlib", reason: "Used only by tests." }],
name: "fake-native",
version: "1.0.0",
});
expect(
manifest.dependencies[0].components.map((component: { role: string }) => component.role),
).toEqual(["build-definition", "compiled", "compiled", "build-input"]);
const metadata = JSON.parse(await readFile(first.metadataPath ?? "", "utf8"));
expect(metadata).toMatchObject({
correspondingSource: {
file: SOURCE_FILE,
sha256: first.sourceSha256,
manifestSha256: first.manifestSha256,
},
covered: [{ name: "fake-native", version: "1.0.0" }],
package: { file: PACKAGE_FILE, sha256: sha256(await readFile(fx.packagePath)) },
});
const notice = await readFile(first.noticePath ?? "", "utf8");
expect(notice).toBe(metadata.notice);
expect(notice).toContain(
"This release includes third-party software covered by licenses including the GNU LGPL",
);
expect(notice).toContain(
"Corresponding source and build/relink materials for the covered native dependencies",
);
expect(notice).toContain(SOURCE_FILE);
expect(notice).toContain(`SHA-256 ${first.sourceSha256}`);
// Nothing machine-specific in what the build generated.
for (const generated of ["README.md", "SHA256SUMS", "corresponding-source.json"]) {
const content =
entries.find((entry) => entry.path === `${TOP}/${generated}`)?.data.toString("utf8") ?? "";
expect(content).not.toContain(fx.root);
expect(content).not.toContain(fx.cache);
}
expect(JSON.stringify(metadata)).not.toContain(fx.root);
});
it("is approved by the gate exactly as built", async () => {
const { fx, metadataPath, sourcePath } = await release();
await expect(verify(fx, sourcePath, metadataPath)).resolves.toMatchObject({
approved: true,
correspondingSource: { file: SOURCE_FILE },
covered: [{ name: "fake-native", version: "1.0.0" }],
});
});
it("refuses to build from cached bytes that differ from the recipe, and writes nothing", async () => {
const fx = await fixture();
await writeFile(
path.join(fx.cache, "sha256", sha256(SOURCES["patches/fakelib-fix.patch"] ?? "")),
"changed patch\n",
);
await rejects(build(fx), "CORRESPONDING_SOURCE_SOURCE_MISMATCH");
await rm(path.join(fx.cache, "sha256", sha256(SOURCES["sources/fakelib-2.0.tar.gz"] ?? "")));
await expect(readFile(path.join(fx.root, "out", SOURCE_FILE))).rejects.toThrow();
});
it("refuses a missing cached file, and never overwrites an existing release", async () => {
const fx = await fixture();
await rm(path.join(fx.cache, "sha256", sha256(SOURCES["sources/helperlib-1.1.tar.gz"] ?? "")));
await rejects(build(fx), "CORRESPONDING_SOURCE_SOURCE_MISSING");
const fresh = await fixture();
await build(fresh);
await rejects(build(fresh), "CORRESPONDING_SOURCE_OUTPUT_EXISTS");
});
it("builds nothing for a package with no covered native code, and the gate approves it without an artifact", async () => {
const fx = await fixture({ license: "MIT", withRecipe: false });
// The fixture's supplement records an LGPL component, so remove it: a package whose own files and
// licence carry no source obligation needs no corresponding source.
await rm(fx.supplementsDirectory, { force: true, recursive: true });
await expect(build(fx)).resolves.toEqual({ required: false });
await expect(verify(fx)).resolves.toMatchObject({ approved: true, covered: [] });
});
it("treats a dependency as covered when its reviewed supplement records a copyleft component", async () => {
// Declared MIT, but the supplement says an LGPL library is built into it.
const fx = await fixture({ license: "MIT", withRecipe: false });
// The supplement's declared licence must match the package's for the supplement to apply.
const supplementPath = path.join(
fx.supplementsDirectory,
"fake-native",
"1.0.0",
"supplement.json",
);
const supplement = JSON.parse(await readFile(supplementPath, "utf8"));
expect(supplement.package.license).toBe("MIT");
await rejects(build(fx), "CORRESPONDING_SOURCE_RECIPE_MISSING");
});
});
describe("the public-distribution gate fails closed", () => {
it("without the corresponding-source archive, or without the redistribution metadata", async () => {
const { fx, metadataPath, sourcePath } = await release();
await rejects(verify(fx), "CORRESPONDING_SOURCE_ARTIFACT_MISSING");
await rm(sourcePath);
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_ARTIFACT_MISSING");
const second = await release();
await rejects(verify(second.fx, second.sourcePath), "CORRESPONDING_SOURCE_METADATA_MISMATCH");
});
it("when the metadata names a different source digest, or the archive's bytes are corrupted", async () => {
const { fx, metadataPath, sourcePath } = await release();
const metadata = JSON.parse(await readFile(metadataPath, "utf8"));
metadata.correspondingSource.sha256 = "0".repeat(64);
await writeFile(metadataPath, JSON.stringify(metadata));
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_METADATA_MISMATCH");
const corrupt = await release();
const bytes = await readFile(corrupt.sourcePath);
const middle = Math.floor(bytes.length / 2);
bytes[middle] = (bytes[middle] ?? 0) ^ 0xff;
await writeFile(corrupt.sourcePath, bytes);
await rejects(
verify(corrupt.fx, corrupt.sourcePath, corrupt.metadataPath),
"CORRESPONDING_SOURCE_ARTIFACT_INVALID",
);
});
it("when the release notice does not name the exact artifact", async () => {
const { fx, metadataPath, sourcePath } = await release();
const metadata = JSON.parse(await readFile(metadataPath, "utf8"));
metadata.notice = metadata.notice.replace(SOURCE_FILE, "some-other-source.tar.gz");
await writeFile(metadataPath, JSON.stringify(metadata));
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_METADATA_MISMATCH");
});
it("when the native package's bytes are not those the recipe was prepared for", async () => {
const built = await release();
// The same name and version, different library bytes: a different content digest.
const other = await fixture({ libraryBytes: "\u007fELF a different build" });
await writeFile(built.fx.packagePath, await readFile(other.packagePath));
await rejects(
verify(built.fx, built.sourcePath, built.metadataPath),
"CORRESPONDING_SOURCE_RECIPE_MISMATCH",
);
});
it("when a source component's version differs from the recipe", async () => {
const { files, fx, metadataPath, relist, sourcePath, write } = await release();
const manifest = JSON.parse(files.get("corresponding-source.json")?.toString("utf8") ?? "{}");
manifest.dependencies[0].components[1].version = "2.1";
files.set("corresponding-source.json", Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`));
relist(files);
await write(files);
const error = await rejects(
verify(fx, sourcePath, metadataPath),
"CORRESPONDING_SOURCE_MANIFEST_MISMATCH",
);
expect(error.subject).toBe("[email protected]");
});
it("when a required source component is missing", async () => {
const { files, fx, metadataPath, relist, sourcePath, write } = await release();
const manifest = JSON.parse(files.get("corresponding-source.json")?.toString("utf8") ?? "{}");
manifest.dependencies[0].components.splice(2, 1);
files.set("corresponding-source.json", Buffer.from(`${JSON.stringify(manifest, null, 2)}\n`));
files.delete("dependencies/fake-native/1.0.0/sources/helperlib-1.1.tar.gz");
relist(files);
await write(files);
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_MANIFEST_MISMATCH");
});
it("when a patch's bytes changed, even with the file list rewritten to match", async () => {
const { files, fx, metadataPath, relist, sourcePath, write } = await release();
files.set(
"dependencies/fake-native/1.0.0/patches/fakelib-fix.patch",
Buffer.from("a different patch\n"),
);
relist(files);
await write(files);
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_MANIFEST_MISMATCH");
});
it("when the manifest records a file the archive does not hold", async () => {
const { files, fx, metadataPath, sourcePath, write } = await release();
files.delete("dependencies/fake-native/1.0.0/sources/fakelib-2.0.tar.gz");
await write(files);
const error = await rejects(
verify(fx, sourcePath, metadataPath),
"CORRESPONDING_SOURCE_MANIFEST_MISMATCH",
);
expect(error.subject).toBe("dependencies/fake-native/1.0.0/sources/fakelib-2.0.tar.gz");
});
it("when the archive holds source material its manifest does not record", async () => {
const { files, fx, metadataPath, sourcePath, write } = await release();
files.set(
"dependencies/fake-native/1.0.0/sources/unreviewed.tar.gz",
Buffer.from("not recorded\n"),
);
await write(files);
const error = await rejects(
verify(fx, sourcePath, metadataPath),
"CORRESPONDING_SOURCE_UNRECORDED_FILE",
);
expect(error.subject).toBe("dependencies/fake-native/1.0.0/sources/unreviewed.tar.gz");
});
it("when a generated file names a machine-specific path", async () => {
const { files, fx, metadataPath, relist, sourcePath, write } = await release();
files.set("README.md", Buffer.from("Built from /home/builder/cache/sha256.\n"));
relist(files);
await write(files);
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_LOCAL_PATH");
});
it("when the archive's entries are not in the deterministic order", async () => {
const { files, fx, metadataPath, sourcePath, write } = await release();
// The same entries and bytes, with the last two file entries swapped.
const raw = gunzipSync(await readFile(sourcePath));
const chunks: Buffer[] = [];
let offset = 0;
while (offset < raw.length && !raw.subarray(offset, offset + 512).every((byte) => byte === 0)) {
const size = Number.parseInt(
raw
.subarray(offset + 124, offset + 136)
.toString("latin1")
.replace(/\0.*$/u, "")
.trim() || "0",
8,
);
const length = 512 + Math.ceil(size / 512) * 512;
chunks.push(raw.subarray(offset, offset + length));
offset += length;
}
const last = chunks.length - 1;
[chunks[last - 1], chunks[last]] = [chunks[last] as Buffer, chunks[last - 1] as Buffer];
await write(files, gzipSync(Buffer.concat([...chunks, Buffer.alloc(1024)]), { level: 9 }));
await rejects(verify(fx, sourcePath, metadataPath), "CORRESPONDING_SOURCE_ARTIFACT_INVALID");
});
it("when the artifact was built for another version of the native dependency", async () => {
// The 1.3.3-for-1.3.4 case: an artifact for fake-native 1.0.0 beside a package with 1.0.1.
const old = await release();
const current = await fixture({ version: "1.0.1" });
await mkdir(path.join(current.recipesDirectory, "fake-native", "1.0.0"), { recursive: true });
for (const file of ["recipe.json", "BUILDING.md"]) {
await writeFile(
path.join(current.recipesDirectory, "fake-native", "1.0.0", file),
await readFile(path.join(old.fx.recipesDirectory, "fake-native", "1.0.0", file)),
);
}
await rejects(
verify(current, old.sourcePath, old.metadataPath),
"CORRESPONDING_SOURCE_MANIFEST_MISMATCH",
);
});
it("when the package bundles copyleft code, which this mechanism does not cover", async () => {
const notices =
noticeEntry("fake-native", "1.0.0", "LGPL-2.1-or-later", "node_modules") +
noticeEntry("bundled-lgpl", "3.0.0", "LGPL-3.0-only", "backend");
const fx = await fixture({ notices });
await rejects(build(fx), "CORRESPONDING_SOURCE_BUNDLED_COPYLEFT");
await rejects(verify(fx), "CORRESPONDING_SOURCE_BUNDLED_COPYLEFT");
});
it("when a covered dependency has no recipe, or its recipe was prepared against another supplement", async () => {
await rejects(
build(await fixture({ withRecipe: false })),
"CORRESPONDING_SOURCE_RECIPE_MISSING",
);
const fx = await fixture({
recipe: (recipe) => {
(recipe.supplement as { sha256: string }).sha256 = "f".repeat(64);
},
});
await rejects(build(fx), "CORRESPONDING_SOURCE_RECIPE_MISMATCH");
const unknownMaterial = await fixture({
recipe: (recipe) => {
const components = recipe.components as {
licenseMaterial: { name: string; version: string }[];
}[];
(components[1] as { licenseMaterial: unknown[] }).licenseMaterial = [
{ name: "fakelib", version: "9.9" },
];
},
});
await rejects(build(unknownMaterial), "CORRESPONDING_SOURCE_RECIPE_MISMATCH");
});
});
describe("source recipes are reviewed inputs", () => {
async function loadWith(change: (recipe: Record<string, unknown>) => void) {
const fx = await fixture({ recipe: change });
return loadSourceRecipes(fx.recipesDirectory);
}
it("refuses an absolute or climbing source path, a non-https origin, or a compiled component without licence material", async () => {
const component = (recipe: Record<string, unknown>, index: number) =>
(recipe.components as Record<string, unknown>[])[index] as Record<string, unknown>;
const first = (recipe: Record<string, unknown>) =>
(component(recipe, 1).sources as Record<string, unknown>[])[0] as Record<string, unknown>;
for (const change of [
(recipe: Record<string, unknown>) => {
first(recipe).file = "/home/builder/fakelib-2.0.tar.gz";
},
(recipe: Record<string, unknown>) => {
first(recipe).file = "sources/../../fakelib-2.0.tar.gz";
},
(recipe: Record<string, unknown>) => {
first(recipe).origin = "http://example.org/fakelib-2.0.tar.gz";
},
(recipe: Record<string, unknown>) => {
component(recipe, 1).licenseMaterial = [];
},
(recipe: Record<string, unknown>) => {
component(recipe, 2).version = "2.0";
component(recipe, 2).name = "fakelib";
},
]) {
await rejects(loadWith(change), "CORRESPONDING_SOURCE_RECIPE_INVALID");
}
});
it("refuses a document whose bytes changed, and an unlisted file beside the recipe", async () => {
const fx = await fixture();
const directory = path.join(fx.recipesDirectory, "fake-native", "1.0.0");
await writeFile(path.join(directory, "BUILDING.md"), "edited without review\n");
await rejects(loadSourceRecipes(fx.recipesDirectory), "CORRESPONDING_SOURCE_RECIPE_INVALID");
const second = await fixture();
await writeFile(
path.join(second.recipesDirectory, "fake-native", "1.0.0", "notes.txt"),
"stray\n",
);
await rejects(
loadSourceRecipes(second.recipesDirectory),
"CORRESPONDING_SOURCE_RECIPE_INVALID",
);
});
it("recognizes licences with source obligations, by SPDX identifier or by upstream name", () => {
for (const licence of [
"LGPL-3.0-or-later",
"MPL-2.0",
"GPL-2.0-only",
"Mozilla Public License 1.1",
"MIT AND LGPL-2.1-or-later",
"EPL-2.0",
]) {
expect(hasSourceObligations(licence), licence).toBe(true);
}
for (const licence of [
"MIT",
"Apache-2.0",
"BSD-3-Clause",
"ISC",
"Zlib",
"MIT OR Apache-2.0",
]) {
expect(hasSourceObligations(licence), licence).toBe(false);
}
});
});
describe("acquisition is separate, verified, and the only network step", () => {
it("fetches each recorded file once, verifies it, and refuses a response that differs", async () => {
const fx = await fixture();
const fetched: string[] = [];
const served = new Map(
Object.entries(SOURCES).map(([file, content]) => [
`https://example.org/${path.basename(file)}`,
content,
]),
);
const empty = path.join(fx.root, "fresh-cache");
const result = await acquireCorrespondingSources({
cacheDirectory: empty,
fetch: async (url) => {
fetched.push(url);
return Buffer.from(served.get(url) ?? "");
},
packageArchivePath: fx.packagePath,
recipesDirectory: fx.recipesDirectory,
supplementsDirectory: fx.supplementsDirectory,
});
expect(result).toMatchObject({ fetched: 5, files: 5 });
// A second run finds everything verified and fetches nothing.
const again = await acquireCorrespondingSources({
cacheDirectory: empty,
fetch: async () => {
throw new Error("no network expected");
},
recipes: [{ name: "fake-native", version: "1.0.0" }],
recipesDirectory: fx.recipesDirectory,
});
expect(again).toMatchObject({ fetched: 0, files: 5 });
// The built artifact from the acquired cache is identical to one built from the original cache.
const fromAcquired = await buildCorrespondingSource({
cacheDirectory: empty,
outDirectory: path.join(fx.root, "acquired-out"),
packageArchivePath: fx.packagePath,
recipesDirectory: fx.recipesDirectory,
supplementsDirectory: fx.supplementsDirectory,
});
expect(fromAcquired.sourceSha256).toBe((await build(fx)).sourceSha256);
// A transient upstream failure is retried; only the recorded bytes are ever stored.
const flaky = path.join(fx.root, "flaky-cache");
const attempts = new Map<string, number>();
await expect(
acquireCorrespondingSources({
cacheDirectory: flaky,
fetch: async (url) => {
const count = (attempts.get(url) ?? 0) + 1;
attempts.set(url, count);
if (count === 1) {
throw new Error("connection reset");
}
return Buffer.from(served.get(url) ?? "");
},
recipes: [{ name: "fake-native", version: "1.0.0" }],
recipesDirectory: fx.recipesDirectory,
}),
).resolves.toMatchObject({ fetched: 5 });
const tampered = path.join(fx.root, "tampered-cache");
await rejects(
acquireCorrespondingSources({
cacheDirectory: tampered,
fetch: async (url) => Buffer.from(`${served.get(url) ?? ""}upstream changed\n`),
recipes: [{ name: "fake-native", version: "1.0.0" }],
recipesDirectory: fx.recipesDirectory,
}),
"CORRESPONDING_SOURCE_ACQUISITION_FAILED",
);
const { readdir } = await import("node:fs/promises");
expect(await readdir(path.join(tampered, "sha256"))).toEqual([]);
});
});
@@ -0,0 +1,60 @@
#!/usr/bin/env node
// Builds the repository's real sharp proof as a native Module Package for the hardened verifier
// container test. Run inside deploy/Dockerfile's build stage, where the workspace is already built
// and has exactly the locked dependencies the runtime image was built against.
import { cp, mkdir, mkdtemp, readFile, realpath, rm, symlink, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { packModule } from "@ekklesia/module-pack";
const repositoryRoot = path.resolve(import.meta.dirname, "..", "..", "..", "..");
const outDirectory = process.argv[2];
if (outDirectory === undefined || !path.isAbsolute(outDirectory)) {
throw new Error("An absolute output directory is required.");
}
const temporaryRoot = await mkdtemp(path.join(tmpdir(), "ekklesia-native-verifier-proof-"));
try {
const moduleDirectory = path.join(temporaryRoot, "native-proof");
await cp(
path.join(repositoryRoot, "packages", "module-pack", "tests", "fixtures", "native-module"),
moduleDirectory,
{ recursive: true },
);
const packageFile = path.join(moduleDirectory, "package.json");
const packageJson = JSON.parse(await readFile(packageFile, "utf8"));
packageJson.engines = { node: ">=24.11.0 <25" };
await writeFile(packageFile, `${JSON.stringify(packageJson, null, 2)}\n`);
const modulesDirectory = path.join(moduleDirectory, "node_modules");
await mkdir(path.join(modulesDirectory, "@ekklesia"), { recursive: true });
await symlink(
await realpath(path.join(repositoryRoot, "packages", "module-pack", "node_modules", "sharp")),
path.join(modulesDirectory, "sharp"),
);
await symlink(
await realpath(
path.join(
repositoryRoot,
"packages",
"module-pack",
"node_modules",
"@ekklesia",
"module-sdk",
),
),
path.join(modulesDirectory, "@ekklesia", "module-sdk"),
);
const result = await packModule({
licenseFile: path.join(repositoryRoot, "LICENSE"),
moduleDirectory,
outDirectory,
sourceRevision: "0".repeat(40),
target: "linux-x64-glibc",
});
process.stdout.write(`${path.basename(result.archivePath)}\n`);
} finally {
await rm(temporaryRoot, { force: true, recursive: true });
}
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/module-sdk", "name": "@ekklesia/module-sdk",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
"files": [ "files": [
+1
View File
@@ -37,6 +37,7 @@ export {
isDevicePrincipal, isDevicePrincipal,
isModuleHttpMethod, isModuleHttpMethod,
isModuleRouteBodyMode, isModuleRouteBodyMode,
isModuleStorageError,
isValidModuleStorageKey, isValidModuleStorageKey,
isValidModuleStorageReadUrlFilename, isValidModuleStorageReadUrlFilename,
MODULE_HTTP_BODY_MODES, MODULE_HTTP_BODY_MODES,
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/module-ui-build", "name": "@ekklesia/module-ui-build",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
"files": [ "files": [
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/module-ui-host", "name": "@ekklesia/module-ui-host",
"version": "0.2.1", "version": "0.2.4",
"private": true, "private": true,
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/module-ui-sdk", "name": "@ekklesia/module-ui-sdk",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
"files": [ "files": [
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/typescript-config", "name": "@ekklesia/typescript-config",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"files": [ "files": [
"base.json", "base.json",
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/ui", "name": "@ekklesia/ui",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"type": "module", "type": "module",
"files": [ "files": [
+67 -1
View File
@@ -6,7 +6,9 @@
# - its module storage mount point, /var/lib/ekklesia/modules, exists, is root-owned, and is empty; # - its module storage mount point, /var/lib/ekklesia/modules, exists, is root-owned, and is empty;
# - EKKLESIA_MODULES_DIR names that directory, in the image configuration and in a running process; # - EKKLESIA_MODULES_DIR names that directory, in the image configuration and in a running process;
# - a checkout with a populated modules/ directory builds the same image: the runtime file system # - a checkout with a populated modules/ directory builds the same image: the runtime file system
# of both builds is identical, and no module from that directory enters it. # of both builds is identical, and no module from that directory enters it;
# - the hardened verifier keeps /tmp noexec while a dedicated bounded execution scratch loads and
# runs the real sharp 0.35.5/libvips 1.3.4 native proof package.
# #
# bash scripts/check-image.sh builds both images from this checkout # bash scripts/check-image.sh builds both images from this checkout
# EKKLESIA_CHECK_IMAGE=<image> bash scripts/check-image.sh checks one existing image's contents # EKKLESIA_CHECK_IMAGE=<image> bash scripts/check-image.sh checks one existing image's contents
@@ -103,6 +105,7 @@ context_copy() {
tag_base="ekklesia-ci:${GITHUB_SHA:-local}-${GITHUB_RUN_ID:-0}-${GITHUB_RUN_ATTEMPT:-1}-" tag_base="ekklesia-ci:${GITHUB_SHA:-local}-${GITHUB_RUN_ID:-0}-${GITHUB_RUN_ATTEMPT:-1}-"
generic_image="${tag_base}generic-$$" generic_image="${tag_base}generic-$$"
populated_image="${tag_base}populated-$$" populated_image="${tag_base}populated-$$"
builder_image="${tag_base}native-proof-builder-$$"
# build_image <context> <tag> # build_image <context> <tag>
build_image() { build_image() {
@@ -143,4 +146,67 @@ else
echo "note the two builds' layers differ (a cold build cache); their file systems do not" echo "note the two builds' layers differ (a cold build cache); their file systems do not"
fi fi
step "Proving native package verification with /tmp noexec and a dedicated execution scratch"
built_images+=("$builder_image")
DOCKER_BUILDKIT=1 docker build --quiet --file "$work/generic/deploy/Dockerfile" \
--target build --label "${EKKLESIA_CI_LABEL}=${project}" \
--label "${EKKLESIA_CI_CREATED_LABEL}=$(date +%s)" \
--tag "$builder_image" "$work/generic" >/dev/null || fail "the native proof builder did not build."
mkdir "$work/native-package"
package_archive="$({
docker run --rm --network none --user "$(id -u):$(id -g)" \
-v "$work/native-package:/proof" --entrypoint node "$builder_image" \
packages/module-pack/tests/support/pack-native-verifier-proof.mjs /proof
} | tail -1)" || fail "the real native proof package did not build."
[[ "$package_archive" == native-proof-0.1.0-linux-x64-glibc.tar.gz ]] ||
fail "the native proof builder returned an unexpected archive."
docker run --rm --network none --read-only \
--tmpfs /tmp:rw,nosuid,nodev,noexec,size=64m \
--tmpfs /run/ekklesia-verifier:rw,nosuid,nodev,exec,size=1100m,mode=0700,uid=1000,gid=1000 \
--cap-drop ALL --security-opt no-new-privileges \
-e "PROOF_ARCHIVE=$package_archive" -v "$work/native-package:/package:ro" \
--entrypoint sh "$generic_image" -ec '
fail() { echo "check-image: $*" >&2; exit 1; }
verifier() {
EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR=/run/ekklesia-verifier \
node core/dist/cli/verify-module-package.js "/package/$PROOF_ARCHIVE" \
--sha256-file "/package/$PROOF_ARCHIVE.sha256" "$@"
}
scratch_failure() {
if output="$(EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR="$1" \
node core/dist/cli/verify-module-package.js "/package/$PROOF_ARCHIVE" \
--sha256-file "/package/$PROOF_ARCHIVE.sha256" 2>&1)"; then
fail "the verifier accepted unusable execution scratch $1."
fi
printf "%s\n" "$output" | grep -Fq "[MODULE_PACKAGE_VERIFIER_SCRATCH_UNAVAILABLE]" ||
fail "the verifier did not identify unusable execution scratch $1."
}
grep -Eq " /tmp .*noexec" /proc/mounts || fail "/tmp is not mounted noexec."
verifier --extract-to /tmp/noexec-installed >/tmp/noexec-verify.log
if env -i /usr/local/bin/node --input-type=module --eval \
"await import(\"file:///tmp/noexec-installed/server/index.js\");" \
>/tmp/noexec-load.log 2>&1; then
fail "the native addon unexpectedly loaded from noexec /tmp."
fi
rm -rf /tmp/noexec-installed
scratch_failure /tmp
scratch_failure /missing-verifier-scratch
scratch_failure /package
verifier --extract-to /run/ekklesia-verifier/installed >/tmp/native-verify.log
env -i /usr/local/bin/node --input-type=module --eval \
"const m=await import(\"file:///run/ekklesia-verifier/installed/server/index.js\"); const actual=await m.proof(); const expected={pixels:[255,0,0,255,0,0,255,0,0,255,0,0],png:{format:\"png\",height:3,width:5},resized:{channels:3,height:2,width:2}}; if(JSON.stringify(actual)!==JSON.stringify(expected)) throw new Error(\"sharp proof mismatch\");"
rm -rf /run/ekklesia-verifier/installed
verifier >/tmp/native-verify-a.log 2>&1 & first=$!
verifier >/tmp/native-verify-b.log 2>&1 & second=$!
wait "$first" || { cat /tmp/native-verify-a.log >&2; fail "first concurrent verification failed."; }
wait "$second" || { cat /tmp/native-verify-b.log >&2; fail "second concurrent verification failed."; }
[ -z "$(find /run/ekklesia-verifier -mindepth 1 -print -quit)" ] ||
fail "the verifier left execution scratch behind."
' || fail "the hardened native verifier proof failed."
echo "ok /tmp stayed noexec; native verification, sharp execution, rejection, concurrency, and cleanup passed"
echo "Generic platform image check passed." echo "Generic platform image check passed."
+34 -16
View File
@@ -35,6 +35,10 @@ function required(name) {
return value; return value;
} }
function optional(name) {
return process.env[name] || "";
}
let failures = 0; let failures = 0;
function check(condition, description) { function check(condition, description) {
@@ -462,36 +466,50 @@ async function ownerCookie() {
} }
/** /**
* The installed-module checks (ADR 0048): whether `example` is in the module inventory, and what * The installed-module checks (ADR 0048): whether the selected module is in the UI inventory and,
* its own route returns. Absent, it is in no inventory and its routes are not mounted at all; * when one is supplied, whether its module-owned probe route responds. Core's own `example` fixture
* present, its records route returns exactly the expected values, which proves its tables, its * retains its stronger records assertion, which also proves its database facade.
* dependencies, and the platform's database facade all work from module storage.
*/ */
async function moduleChecks() { async function moduleChecks() {
const expectation = required("SMOKE_MODULE_EXPECT"); const expectation = required("SMOKE_MODULE_EXPECT");
const moduleId = optional("SMOKE_MODULE_ID") || "example";
const probePath = optional("SMOKE_MODULE_PROBE_PATH");
const cookie = await ownerCookie(); const cookie = await ownerCookie();
const headers = { cookie, origin: dashboardOrigin }; const headers = { cookie, origin: dashboardOrigin };
const inventory = await fetch(`${apiOrigin}/api/v1/module-ui`, { headers }); const inventory = await fetch(`${apiOrigin}/api/v1/module-ui`, { headers });
const inventoryBody = await inventory.json(); const inventoryBody = await inventory.json();
const installed = const installed =
inventory.status === 200 && inventoryBody.modules.some((module) => module.id === "example"); inventory.status === 200 && inventoryBody.modules.some((module) => module.id === moduleId);
const records = await fetch(`${apiOrigin}/api/v1/modules/example/records`, { headers });
if (expectation === "absent") { if (expectation === "absent") {
check(inventory.status === 200 && !installed, "the module inventory has no `example` module");
check( check(
records.status === 404 && (await records.text()) === '{"code":"NOT_FOUND"}', inventory.status === 200 && !installed,
"the absent module's routes are not mounted", `the module inventory has no \`${moduleId}\` module`,
); );
} else { } else {
const expected = JSON.parse(expectation); check(installed, `the module inventory lists the \`${moduleId}\` module`);
check(installed, "the module inventory lists the `example` module"); }
const values = records.status === 200 ? (await records.json()).map((row) => row.value) : null;
check( if (moduleId === "example") {
JSON.stringify(values) === JSON.stringify(expected), const records = await fetch(`${apiOrigin}/api/v1/modules/example/records`, { headers });
`the module's records route returns ${JSON.stringify(expected)}`, if (expectation === "absent") {
); check(
records.status === 404 && (await records.text()) === '{"code":"NOT_FOUND"}',
"the absent module's routes are not mounted",
);
} else {
const expected = JSON.parse(expectation);
const values = records.status === 200 ? (await records.json()).map((row) => row.value) : null;
check(
JSON.stringify(values) === JSON.stringify(expected),
`the module's records route returns ${JSON.stringify(expected)}`,
);
}
} else if (expectation !== "absent" && probePath) {
const probe = await fetch(`${apiOrigin}${probePath}`, { headers });
await probe.arrayBuffer();
check(probe.status === 200, `the module probe ${probePath} responds`);
} }
const signOut = await fetch(`${apiOrigin}/api/auth/sign-out`, { const signOut = await fetch(`${apiOrigin}/api/auth/sign-out`, {
+42 -17
View File
@@ -426,23 +426,48 @@ async function seedSignage(owner) {
const base = "/api/v1/modules/signage"; const base = "/api/v1/modules/signage";
const images = []; const images = [];
for (const [index, colour] of ["#b03030", "#3060b0", "#30a050"].entries()) { for (const [index, colour] of ["#b03030", "#3060b0", "#30a050"].entries()) {
const uploaded = await owner.evaluate( let uploaded;
`(async () => { for (let attempt = 1; attempt <= 2; attempt += 1) {
const canvas = document.createElement("canvas"); uploaded = await owner.evaluate(
canvas.width = 640; `(async () => {
canvas.height = 360; const canvas = document.createElement("canvas");
const context = canvas.getContext("2d"); canvas.width = 640;
context.fillStyle = ${JSON.stringify(colour)}; canvas.height = 360;
context.fillRect(0, 0, 640, 360); const context = canvas.getContext("2d");
const blob = await new Promise((resolve) => canvas.toBlob(resolve, "image/png")); context.fillStyle = ${JSON.stringify(colour)};
const response = await fetch(${JSON.stringify(`${apiOrigin}${base}/images?filename=slide-${index}.png&displayName=Slide%20${index}`)}, { context.fillRect(0, 0, 640, 360);
body: blob, credentials: "include", headers: { "content-type": "image/png" }, method: "POST", const blob = await new Promise((resolve) => canvas.toBlob(resolve, "image/png"));
signal: AbortSignal.timeout(${requestTimeoutMs}), return fetch(${JSON.stringify(`${apiOrigin}${base}/images?filename=slide-${index}.png&displayName=Slide%20${index}`)}, {
}); body: blob, credentials: "include", headers: { "content-type": "image/png" }, method: "POST",
return { status: response.status, body: await response.json().catch(() => null) }; signal: AbortSignal.timeout(${requestTimeoutMs}),
})()`, }).then(
`uploading slide ${index}`, async (response) => ({ status: response.status, body: await response.json().catch(() => null) }),
); (error) => ({ status: error && error.name === "TimeoutError" ? "timeout" : "network-error" }),
);
})()`,
`uploading slide ${index}`,
);
if (
attempt === 1 &&
uploaded.status === 503 &&
uploaded.body?.code === "SIGNAGE_STORAGE_UNAVAILABLE"
) {
note(`slide ${index} met one transient storage outage; retrying once`);
await sleep(1_000);
continue;
}
break;
}
if (uploaded.status === "timeout") {
throw new HangError(`HTTP POST ${base}/images from page "owner"`, requestTimeoutMs);
}
if (uploaded.status !== 200 && uploaded.status !== 201) {
const code =
typeof uploaded.body?.code === "string" && /^SIGNAGE_[A-Z_]+$/u.test(uploaded.body.code)
? ` (${uploaded.body.code})`
: "";
throw new Error(`Uploading slide ${index} returned HTTP ${uploaded.status}${code}.`);
}
images.push(uploaded.body?.image?.id); images.push(uploaded.body?.image?.id);
} }
check( check(
+195 -51
View File
@@ -7,6 +7,8 @@
# bash scripts/smoke-deploy.sh builds the image first # bash scripts/smoke-deploy.sh builds the image first
# EKKLESIA_SMOKE_IMAGE=<image> bash scripts/smoke-deploy.sh # EKKLESIA_SMOKE_IMAGE=<image> bash scripts/smoke-deploy.sh
# EKKLESIA_SMOKE_TOPOLOGY=split bash scripts/smoke-deploy.sh # EKKLESIA_SMOKE_TOPOLOGY=split bash scripts/smoke-deploy.sh
# EKKLESIA_SMOKE_MODULE_RELEASE_DIR=<dir> EKKLESIA_SMOKE_MODULE_ID=<id> \
# EKKLESIA_SMOKE_TOPOLOGY=split bash scripts/smoke-deploy.sh
# #
# Either topology (ADR 0045), and in both the same generic image, built once from deploy/Dockerfile. # Either topology (ADR 0045), and in both the same generic image, built once from deploy/Dockerfile.
# It contains no module; module storage starts empty, as on a fresh installation. `same-origin` (the # It contains no module; module storage starts empty, as on a fresh installation. `same-origin` (the
@@ -15,10 +17,10 @@
# ADR 0048 end to end with that same image: # ADR 0048 end to end with that same image:
# #
# - it boots with no modules; # - it boots with no modules;
# - the repository's module template is packaged by the canonical packager as an Ekklesia Module # - either the repository's module template is packaged by the canonical packager, or an external
# Package v1 (ADR 0049; scripts/lib/build-module-package.Dockerfile only provides the build # canonical release pair is supplied through EKKLESIA_SMOKE_MODULE_RELEASE_DIR; the package is
# environment), verified by the same image's own verifier, extracted to staging, installed into # verified by the same image's own verifier, extracted to staging, installed into module storage,
# module storage, its migrations planned and explicitly applied, and the API restarted; # its migrations planned and explicitly applied, and the API restarted;
# - a real headless Chromium drives the Dashboard, the module, its display, and sign-out; # - a real headless Chromium drives the Dashboard, the module, its display, and sign-out;
# - the module is removed and the API restarted: Core still runs, migration planning reports the # - the module is removed and the API restarted: Core still runs, migration planning reports the
# module's retained history, and its data survives; # module's retained history, and its data survives;
@@ -83,6 +85,28 @@ same-origin | split) ;;
;; ;;
esac esac
module_release_directory="${EKKLESIA_SMOKE_MODULE_RELEASE_DIR:-}"
module_id="${EKKLESIA_SMOKE_MODULE_ID:-example}"
module_probe_path="${EKKLESIA_SMOKE_MODULE_PROBE_PATH:-}"
[[ "$module_id" =~ ^[a-z][a-z0-9]*(-[a-z0-9]+)*$ ]] || {
echo "smoke-deploy: EKKLESIA_SMOKE_MODULE_ID is not a valid module id." >&2
exit 1
}
if [[ -n "$module_release_directory" ]]; then
[[ "$topology" == split ]] || {
echo "smoke-deploy: an external module release requires the split topology." >&2
exit 1
}
[[ -d "$module_release_directory" ]] || {
echo "smoke-deploy: EKKLESIA_SMOKE_MODULE_RELEASE_DIR is not a directory." >&2
exit 1
}
fi
if [[ -n "$module_probe_path" && "$module_probe_path" != "/api/v1/modules/${module_id}/"* ]]; then
echo "smoke-deploy: EKKLESIA_SMOKE_MODULE_PROBE_PATH must be inside the selected module's API namespace." >&2
exit 1
fi
run_id="$(openssl rand -hex 4)" run_id="$(openssl rand -hex 4)"
# Unique per invocation, and findable by run and job if this script is killed before its trap runs # Unique per invocation, and findable by run and job if this script is killed before its trap runs
# (scripts/lib/ci-project.sh, scripts/ci-sweep.sh). Everything started outside Compose below carries # (scripts/lib/ci-project.sh, scripts/ci-sweep.sh). Everything started outside Compose below carries
@@ -217,22 +241,31 @@ if [[ -z "$image" ]]; then
fi fi
if [[ "$topology" == split ]]; then if [[ "$topology" == split ]]; then
# The module template as a Module Package v1, from the canonical packager. Never a second platform # Never a second platform image: the package is installed into module storage below, after the
# image: the package is installed into module storage below, after the image verifies it. # generic image verifies it. The default keeps Core's own template smoke self-contained; downstream
step "Packaging the module template with the canonical Ekklesia packager" # module CI hands this harness the exact release pair it built instead.
source_revision="${GITHUB_SHA:-$(git rev-parse HEAD 2>/dev/null || true)}" if [[ -z "$module_release_directory" ]]; then
[[ "$source_revision" =~ ^[0-9a-f]{40}$ ]] || fail "no source revision to record in the package." step "Packaging the module template with the canonical Ekklesia packager"
DOCKER_BUILDKIT=1 docker build --quiet --file scripts/lib/build-module-package.Dockerfile \ source_revision="${GITHUB_SHA:-$(git rev-parse HEAD 2>/dev/null || true)}"
--label "${EKKLESIA_CI_LABEL}=${project}" --build-arg SOURCE_REVISION="$source_revision" \ [[ "$source_revision" =~ ^[0-9a-f]{40}$ ]] || fail "no source revision to record in the package."
--output "type=local,dest=${work}/module-package" . >/dev/null || DOCKER_BUILDKIT=1 docker build --quiet --file scripts/lib/build-module-package.Dockerfile \
fail "the module package did not build." --label "${EKKLESIA_CI_LABEL}=${project}" --build-arg SOURCE_REVISION="$source_revision" \
--output "type=local,dest=${work}/module-package" . >/dev/null ||
fail "the module package did not build."
package_output="$work/module-package"
else
step "Accepting the external module's canonical release pair"
package_output="$module_release_directory"
fi
# The archive and its digest are one release: copied together, from this run's own build output, # The archive and its digest are one release: copied together, from this run's own build output,
# into a fresh directory the verifier's unprivileged user can read. BuildKit's output directory is # into a fresh directory the verifier's unprivileged user can read. BuildKit's output directory is
# mode 0700 and owned by whoever built it, so it is never handed to the verifier itself. # mode 0700 and owned by whoever built it, so it is never handed to the verifier itself.
package_archive="$(module_package_release "$work/module-package" "$work/package")" || package_archive="$(module_package_release "$package_output" "$work/package")" ||
fail "the packager did not produce exactly one archive and its digest." fail "the packager did not produce exactly one archive and its digest."
[[ "$package_archive" =~ ^example-[0-9]+\.[0-9]+\.[0-9]+-any\.tar\.gz$ ]] || if [[ -z "$module_release_directory" ]]; then
fail "the packager produced an unexpected archive name." [[ "$package_archive" =~ ^example-[0-9]+\.[0-9]+\.[0-9]+-any\.tar\.gz$ ]] ||
fail "the packager produced an unexpected archive name."
fi
echo "ok packaged ${package_archive} ($(wc -c <"$work/package/$package_archive") bytes, sha256 $(cut -d' ' -f1 "$work/package/${package_archive}.sha256"))" echo "ok packaged ${package_archive} ($(wc -c <"$work/package/$package_archive") bytes, sha256 $(cut -d' ' -f1 "$work/package/${package_archive}.sha256"))"
fi fi
@@ -446,22 +479,48 @@ module_storage() {
install_test_module() { install_test_module() {
local verified local verified
verified="$( verified="$(
docker run --rm --network none --read-only --tmpfs /tmp --cap-drop ALL \ docker run --rm --network none --read-only \
--security-opt no-new-privileges \ --tmpfs /tmp:rw,nosuid,nodev,noexec,size=64m \
--tmpfs /run/ekklesia-verifier:rw,nosuid,nodev,exec,size=1100m,mode=0700,uid=1000,gid=1000 \
--cap-drop ALL --security-opt no-new-privileges \
-e EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR=/run/ekklesia-verifier \
-v "$daemon_work/package:/package:ro" -v "$daemon_work/staging:/staging" \ -v "$daemon_work/package:/package:ro" -v "$daemon_work/staging:/staging" \
--entrypoint node "$image" core/dist/cli/verify-module-package.js "/package/${package_archive}" \ --entrypoint node "$image" core/dist/cli/verify-module-package.js "/package/${package_archive}" \
--sha256-file "/package/${package_archive}.sha256" --extract-to /staging/example 2>&1 --sha256-file "/package/${package_archive}.sha256" --extract-to "/staging/${module_id}" 2>&1
)" || fail "the module package did not verify: ${verified}" )" || fail "the module package did not verify: ${verified}"
echo "ok ${verified%%$'\n'*}" echo "ok ${verified%%$'\n'*}"
docker run --rm --network none --entrypoint sh -v "$daemon_work/modules:/modules" \ docker run --rm --network none --entrypoint sh -v "$daemon_work/modules:/modules" \
-v "$daemon_work/staging:/staging" "$postgres_image" -ec ' -v "$daemon_work/staging:/staging" -e MODULE_ID="$module_id" "$postgres_image" -ec '
rm -rf /modules/.incoming-example && cp -R /staging/example /modules/.incoming-example && incoming="/modules/.incoming-${MODULE_ID}"
rm -rf /staging/example && chown -R 0:0 /modules/.incoming-example && installed="/modules/${MODULE_ID}"
chmod -R a+rX,go-w /modules/.incoming-example && staged="/staging/${MODULE_ID}"
mv /modules/.incoming-example /modules/example' || rm -rf "$incoming" && cp -R "$staged" "$incoming" && rm -rf "$staged" &&
chown -R 0:0 "$incoming" && chmod -R a+rX,go-w "$incoming" && mv "$incoming" "$installed"' ||
fail "the verified module could not be installed into module storage." fail "the verified module could not be installed into module storage."
} }
# The package and its claimed digest are deliberately made to disagree. The verifier must reject it
# before extraction; the valid release pair is left untouched for the installation that follows.
reject_invalid_package() {
local rejected zero_digest
zero_digest="$(printf '0%.0s' {1..64})"
if rejected="$(
docker run --rm --network none --read-only \
--tmpfs /tmp:rw,nosuid,nodev,noexec,size=64m \
--tmpfs /run/ekklesia-verifier:rw,nosuid,nodev,exec,size=1100m,mode=0700,uid=1000,gid=1000 \
--cap-drop ALL --security-opt no-new-privileges \
-e EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR=/run/ekklesia-verifier \
-v "$daemon_work/package:/package:ro" \
--entrypoint node "$image" core/dist/cli/verify-module-package.js "/package/${package_archive}" \
--sha256 "$zero_digest" 2>&1
)"; then
fail "the verifier accepted a package whose archive did not match its claimed digest."
fi
grep -q '^\[MODULE_PACKAGE_DIGEST_MISMATCH\]' <<<"$rejected" ||
fail "the invalid package was rejected for an unexpected reason."
echo "ok an invalid package is rejected before installation"
}
# restart_api <command...>: stops the API, runs the command, and starts the API again. A controlled # restart_api <command...>: stops the API, runs the command, and starts the API again. A controlled
# restart is the only way a module change is picked up; nothing is ever hot loaded. # restart is the only way a module change is picked up; nothing is ever hot loaded.
restart_api() { restart_api() {
@@ -473,9 +532,21 @@ restart_api() {
fi fi
} }
# recreate_api removes the application container, not its installation-owned module bind mount or
# database volumes, then creates a fresh container from the same generic image.
recreate_api() {
"${compose[@]}" stop api >/dev/null 2>&1 || fail "the API did not stop for recreation."
"${compose[@]}" rm --force api >/dev/null 2>&1 || fail "the API container was not removed."
if ! up_output="$("${compose[@]}" up --detach --wait --wait-timeout 180 api 2>&1)"; then
compose_report_not_ready "$up_output" "${compose[@]}"
fail "the recreated API did not become healthy."
fi
}
# module_check <expectation>: "absent", or the exact JSON array of values `example` must return. # module_check <expectation>: "absent", or the exact JSON array of values `example` must return.
module_check() { module_check() {
client module --ip "$module_client_address" -e SMOKE_MODULE_EXPECT="$1" \ client module --ip "$module_client_address" -e SMOKE_MODULE_EXPECT="$1" \
-e SMOKE_MODULE_ID="$module_id" -e SMOKE_MODULE_PROBE_PATH="$module_probe_path" \
-e SMOKE_OWNER_EMAIL="$owner_email" -e SMOKE_OWNER_PASSWORD="$owner_password" -e SMOKE_OWNER_EMAIL="$owner_email" -e SMOKE_OWNER_PASSWORD="$owner_password"
} }
@@ -488,21 +559,36 @@ if [[ "$topology" == split ]]; then
step "Booting the generic image with empty module storage" step "Booting the generic image with empty module storage"
module_check absent || fail "the image without modules did not run module-free." module_check absent || fail "the image without modules did not run module-free."
step "Rejecting an invalid package before installation"
reject_invalid_package
step "Installing the test module into module storage, migrating explicitly, and restarting the API" step "Installing the test module into module storage, migrating explicitly, and restarting the API"
install_module_and_migrate() { install_module_and_migrate() {
install_test_module install_test_module
local plan local plan
plan="$(plan_output)" plan="$(plan_output)"
grep -q '^- \[pending\] example:0000_initial ' <<<"$plan" || grep -q "^- \\[pending\\] ${module_id}:" <<<"$plan" ||
fail "the plan does not list the installed module's migration as pending." fail "the plan does not list the installed module's migration as pending."
"${compose[@]}" run --rm --no-deps migrate node core/dist/cli/apply-migrations.js >/dev/null || "${compose[@]}" run --rm --no-deps migrate node core/dist/cli/apply-migrations.js >/dev/null ||
fail "applying the installed module's migration failed." fail "applying the installed module's migration failed."
plan="$(plan_output)"
grep -q "^- \\[applied\\] ${module_id}:" <<<"$plan" ||
fail "the plan does not list the installed module's migration as applied."
if grep -q "^- \\[pending\\] ${module_id}:" <<<"$plan"; then
fail "the installed module still has a pending migration after apply."
fi
"${compose[@]}" run --rm --no-deps migrate node core/dist/cli/apply-migrations.js >/dev/null ||
fail "reapplying migrations was not idempotent."
} }
restart_api install_module_and_migrate restart_api install_module_and_migrate
module_check '[]' || fail "the installed module did not run from module storage." if [[ "$module_id" == example ]]; then
psql_as_superuser "INSERT INTO example_records (id, value) VALUES (gen_random_uuid(), 'kept')" \ module_check '[]' || fail "the installed module did not run from module storage."
>/dev/null || fail "could not write the module's own data." psql_as_superuser "INSERT INTO example_records (id, value) VALUES (gen_random_uuid(), 'kept')" \
module_check '["Hello, kept"]' || fail "the installed module did not read its own data." >/dev/null || fail "could not write the module's own data."
module_check '["Hello, kept"]' || fail "the installed module did not read its own data."
else
module_check present || fail "the installed module did not run from module storage."
fi
step "Driving a real browser through the split Dashboard, a module, its display, and sign-out" step "Driving a real browser through the split Dashboard, a module, its display, and sign-out"
# Chromium trusts this run's proxy through the public-key hash of Caddy's internal intermediate CA, # Chromium trusts this run's proxy through the public-key hash of Caddy's internal intermediate CA,
@@ -513,24 +599,31 @@ if [[ "$topology" == split ]]; then
spki="$(openssl x509 -in "$work/caddy-intermediate.crt" -pubkey -noout | spki="$(openssl x509 -in "$work/caddy-intermediate.crt" -pubkey -noout |
openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | openssl base64 -A)" openssl pkey -pubin -outform der | openssl dgst -sha256 -binary | openssl base64 -A)"
browser_container="${project}-browser" browser_container="${project}-browser"
# The image's entry script passes its arguments on unquoted, so none may contain a space: the start_owner_browser() {
# hostnames reach Chromium through /etc/hosts, which it reads, rather than a resolver rule. docker rm --force "$browser_container" >/dev/null 2>&1 || true
docker run --detach --rm --name "$browser_container" --label "${EKKLESIA_CI_LABEL}=${project}" \ # The image's entry script passes its arguments on unquoted, so none may contain a space: the
--cap-drop ALL --security-opt no-new-privileges --tmpfs /tmp \ # hostnames reach Chromium through /etc/hosts, which it reads, rather than a resolver rule.
--network "$edge_network" --ip "$browser_address" "${host_mappings[@]}" \ docker run --detach --rm --name "$browser_container" --label "${EKKLESIA_CI_LABEL}=${project}" \
"$browser_image" \ --cap-drop ALL --security-opt no-new-privileges --tmpfs /tmp \
"--ignore-certificate-errors-spki-list=${spki}" \ --network "$edge_network" --ip "$browser_address" "${host_mappings[@]}" \
--user-data-dir=/tmp/profile >/dev/null || fail "the headless browser did not start." "$browser_image" \
"--ignore-certificate-errors-spki-list=${spki}" \
--user-data-dir=/tmp/profile >/dev/null || fail "the headless browser did not start."
}
cp scripts/lib/deploy-smoke-browser.mjs "$work/browser.mjs" cp scripts/lib/deploy-smoke-browser.mjs "$work/browser.mjs"
stage browser.mjs stage browser.mjs
docker run --rm --read-only --cap-drop ALL --security-opt no-new-privileges \ run_browser_checks() {
--label "${EKKLESIA_CI_LABEL}=${project}" \ start_owner_browser
--network "$edge_network" \ docker run --rm --read-only --cap-drop ALL --security-opt no-new-privileges \
-e SMOKE_CDP_ADDRESS="${browser_address}:9222" \ --label "${EKKLESIA_CI_LABEL}=${project}" \
-e SMOKE_HOSTNAME="$hostname" -e SMOKE_DASHBOARD_HOSTNAME="$dashboard_hostname" \ --network "$edge_network" \
-e SMOKE_OWNER_EMAIL="$owner_email" -e SMOKE_OWNER_PASSWORD="$owner_password" \ -e SMOKE_CDP_ADDRESS="${browser_address}:9222" \
-v "$daemon_work/browser.mjs:/smoke/browser.mjs:ro" \ -e SMOKE_HOSTNAME="$hostname" -e SMOKE_DASHBOARD_HOSTNAME="$dashboard_hostname" \
--entrypoint node "$image" /smoke/browser.mjs || fail "browser checks failed." -e SMOKE_OWNER_EMAIL="$owner_email" -e SMOKE_OWNER_PASSWORD="$owner_password" \
-v "$daemon_work/browser.mjs:/smoke/browser.mjs:ro" \
--entrypoint node "$image" /smoke/browser.mjs
}
run_browser_checks || fail "browser checks failed."
if [[ -n "${EKKLESIA_SMOKE_DISPLAY_DEVICE:-}" ]]; then if [[ -n "${EKKLESIA_SMOKE_DISPLAY_DEVICE:-}" ]]; then
# A kiosk: a second browser whose profile directory is a volume, so it outlives the container # A kiosk: a second browser whose profile directory is a volume, so it outlives the container
@@ -566,6 +659,7 @@ if [[ "$topology" == split ]]; then
--network "$edge_network" \ --network "$edge_network" \
-e SMOKE_CDP_ADDRESS="${browser_address}:9222" \ -e SMOKE_CDP_ADDRESS="${browser_address}:9222" \
-e SMOKE_KIOSK_CDP_ADDRESS="${kiosk_address}:9222" \ -e SMOKE_KIOSK_CDP_ADDRESS="${kiosk_address}:9222" \
-e SMOKE_CALL_TIMEOUT_MS="${EKKLESIA_SMOKE_CALL_TIMEOUT_MS:-30000}" \
-e SMOKE_STEP_BUDGET_MS="$((budget * 1000))" \ -e SMOKE_STEP_BUDGET_MS="$((budget * 1000))" \
-e SMOKE_WATCH_MS="${EKKLESIA_SMOKE_WATCH_MS:-120000}" \ -e SMOKE_WATCH_MS="${EKKLESIA_SMOKE_WATCH_MS:-120000}" \
-e SMOKE_RECYCLE_WATCH_MS="${EKKLESIA_SMOKE_RECYCLE_WATCH_MS:-330000}" \ -e SMOKE_RECYCLE_WATCH_MS="${EKKLESIA_SMOKE_RECYCLE_WATCH_MS:-330000}" \
@@ -607,29 +701,79 @@ if [[ "$topology" == split ]]; then
((display_status == 0)) || fail "display-device checks failed." ((display_status == 0)) || fail "display-device checks failed."
fi fi
step "Restarting the API process with the installed module and data intact"
restart_api
if [[ "$module_id" == example ]]; then
module_check '["Hello, kept"]' || fail "the module or its data did not survive an API restart."
else
module_check present || fail "the module did not survive an API restart."
fi
run_browser_checks || fail "the module UI did not survive an API restart."
step "Recreating the API container with the same module storage and database volumes"
recreate_api
if [[ "$module_id" == example ]]; then
module_check '["Hello, kept"]' || fail "the module or its data did not survive container recreation."
else
module_check present || fail "the module did not survive container recreation."
fi
plan="$(plan_output)"
if grep -q "^- \\[pending\\] ${module_id}:" <<<"$plan"; then
fail "container recreation made an applied module migration pending again."
fi
run_browser_checks || fail "the module UI did not survive container recreation."
if [[ -n "${EKKLESIA_SMOKE_DISPLAY_DEVICE:-}" ]]; then
display_driver 120 restart || fail "the display runtime or its data did not survive container recreation."
fi
step "Disabling and re-enabling the module through the supported lifecycle command"
restart_api "${compose[@]}" run --rm --no-deps migrate node \
core/dist/cli/set-module-enabled.js --disable --module "$module_id"
module_check absent || fail "the disabled module was still reported as running."
module_storage "test -d /modules/${module_id}" ||
fail "disabling the module removed its installed package."
restart_api "${compose[@]}" run --rm --no-deps migrate node \
core/dist/cli/set-module-enabled.js --enable --module "$module_id"
if [[ "$module_id" == example ]]; then
module_check '["Hello, kept"]' || fail "the re-enabled module did not return with its data."
else
module_check present || fail "the re-enabled module did not return."
fi
run_browser_checks || fail "the re-enabled module UI did not return."
step "Removing the module from module storage and restarting: Core runs, its data stays" step "Removing the module from module storage and restarting: Core runs, its data stays"
restart_api module_storage 'rm -rf /modules/example' restart_api module_storage "rm -rf /modules/${module_id}"
module_check absent || fail "the removed module is still running." module_check absent || fail "the removed module is still running."
plan="$(plan_output)" plan="$(plan_output)"
grep -q '^- \[retained\] example: 1 applied migration kept; the module is not installed$' \ grep -q "^- \\[retained\\] ${module_id}: .*applied migration.*kept; the module is not installed$" \
<<<"$plan" || fail "migration planning did not report the removed module's retained history." <<<"$plan" || fail "migration planning did not report the removed module's retained history."
grep -q '^Migration plan (.*0 pending, 0 approval-blocked, 1 retained):$' <<<"$plan" || grep -q '^Migration plan (.*0 pending, 0 approval-blocked, 1 retained):$' <<<"$plan" ||
fail "migration planning did not succeed cleanly with the module absent." fail "migration planning did not succeed cleanly with the module absent."
"${compose[@]}" run --rm --no-deps migrate node core/dist/cli/apply-migrations.js >/dev/null || "${compose[@]}" run --rm --no-deps migrate node core/dist/cli/apply-migrations.js >/dev/null ||
fail "applying migrations failed with the module absent." fail "applying migrations failed with the module absent."
[[ "$(psql_as_superuser "SELECT value FROM example_records")" == "kept" ]] || if [[ "$module_id" == example ]]; then
fail "the removed module's data did not survive." [[ "$(psql_as_superuser "SELECT value FROM example_records")" == "kept" ]] ||
fail "the removed module's data did not survive."
fi
echo "ok the module is gone, Core runs, its history is retained, and its data survives" echo "ok the module is gone, Core runs, its history is retained, and its data survives"
step "Reinstalling the exact same package and restarting: the module returns with its previous data" step "Reinstalling the exact same package and restarting: the module returns with its previous data"
restart_api install_test_module restart_api install_test_module
plan="$(plan_output)" plan="$(plan_output)"
grep -q '^- \[applied\] example:0000_initial ' <<<"$plan" || grep -q "^- \\[applied\\] ${module_id}:" <<<"$plan" ||
fail "the restored module's migration history is not valid." fail "the restored module's migration history is not valid."
if grep -q '\[retained\]\|\[pending\]' <<<"$plan"; then if grep -q '\[retained\]\|\[pending\]' <<<"$plan"; then
fail "the restored module left retained or pending migrations." fail "the restored module left retained or pending migrations."
fi fi
module_check '["Hello, kept"]' || fail "the restored module did not return with its data." if [[ "$module_id" == example ]]; then
module_check '["Hello, kept"]' || fail "the restored module did not return with its data."
else
module_check present || fail "the restored module did not return."
fi
run_browser_checks || fail "the restored module UI did not return."
if [[ -n "${EKKLESIA_SMOKE_DISPLAY_DEVICE:-}" ]]; then
display_driver 120 restart || fail "the restored display runtime did not return with its data."
fi
echo "ok the restored module runs again, with its history strict and its data intact" echo "ok the restored module runs again, with its history strict and its data intact"
fi fi
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
# ADR 0055's full proof, with real sources: the native proof module (sharp 0.35.5, libvips 1.3.4) and
# its corresponding-source artifact.
#
# bash scripts/test-corresponding-source.sh
#
# 1. Acquisition (network, first run only): every checked-in recipe's recorded source files are
# downloaded into a verified cache, each checked against its recorded size and SHA-256. A file
# already in the cache with the right bytes is never fetched again.
# 2. Construction and the gate (offline): the artifact is built twice from unrelated directories and
# must be byte-identical; the public-distribution gate must approve it, and must refuse it once it
# is removed or corrupted; sharp must still run from the verified binary package.
#
# EKKLESIA_CORRESPONDING_SOURCE_CACHE chooses the cache directory; the default is
# ${XDG_CACHE_HOME:-$HOME/.cache}/ekklesia/corresponding-sources. It is about 200 MB per recipe.
set -Eeuo pipefail
root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$root"
cache="${EKKLESIA_CORRESPONDING_SOURCE_CACHE:-${XDG_CACHE_HOME:-$HOME/.cache}/ekklesia/corresponding-sources}"
pnpm exec turbo run build --filter=@ekklesia/module-pack... --filter=@ekklesia/core... >/dev/null
recipes="packages/module-pack/corresponding-sources"
while IFS= read -r recipe; do
relative="${recipe#"$recipes"/}"
relative="${relative%/recipe.json}"
echo "==> Acquiring the recorded sources for ${relative%/*}@${relative##*/}"
node packages/module-pack/bin/ekklesia-corresponding-source.mjs acquire \
--recipe "${relative%/*}@${relative##*/}" --cache "$cache"
done < <(find "$recipes" -name recipe.json | LC_ALL=C sort)
echo "==> Building, gating, and running the native proof offline"
EKKLESIA_CORRESPONDING_SOURCE_CACHE="$cache" EKKLESIA_CORRESPONDING_SOURCE_REQUIRED=1 \
pnpm --filter @ekklesia/module-pack exec vitest run tests/corresponding-source-native.test.ts
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "@ekklesia/shell", "name": "@ekklesia/shell",
"version": "0.2.1", "version": "0.2.4",
"license": "AGPL-3.0-or-later", "license": "AGPL-3.0-or-later",
"private": true, "private": true,
"type": "module", "type": "module",
+41 -3
View File
@@ -1,5 +1,5 @@
import { designToken } from "@ekklesia/design-tokens"; import { designToken } from "@ekklesia/design-tokens";
import { Button, Field, TextInput } from "@ekklesia/ui"; import { Button, Dialog, Field, TextInput } from "@ekklesia/ui";
import { type FormEvent, type ReactElement, useCallback, useEffect, useState } from "react"; import { type FormEvent, type ReactElement, useCallback, useEffect, useState } from "react";
import { import {
approveDevicePairing, approveDevicePairing,
@@ -61,6 +61,10 @@ export function DevicePanel(props: DevicePanelProps): ReactElement | undefined {
const [code, setCode] = useState(""); const [code, setCode] = useState("");
const [name, setName] = useState(""); const [name, setName] = useState("");
const [pairing, setPairing] = useState<PairingStep>({ step: "idle" }); const [pairing, setPairing] = useState<PairingStep>({ step: "idle" });
/** The device an administrator asked to revoke and has not yet confirmed. */
const [confirming, setConfirming] = useState<HostDeviceSummary | undefined>(undefined);
const [revoking, setRevoking] = useState(false);
const dismissConfirmation = useCallback(() => setConfirming(undefined), []);
const refresh = useCallback(async (): Promise<void> => { const refresh = useCallback(async (): Promise<void> => {
try { try {
@@ -106,13 +110,15 @@ export function DevicePanel(props: DevicePanelProps): ReactElement | undefined {
}; };
const revoke = async (deviceId: string): Promise<void> => { const revoke = async (deviceId: string): Promise<void> => {
setRevoking(true);
try { try {
await revokeHostDevice(config, deviceId); await revokeHostDevice(config, deviceId);
await refresh();
} catch { } catch {
// Left to the next refresh rather than asserting a failure the operator cannot act on. // Left to the next refresh rather than asserting a failure the operator cannot act on.
} }
await refresh(); await refresh();
setRevoking(false);
setConfirming(undefined);
}; };
return ( return (
@@ -195,7 +201,7 @@ export function DevicePanel(props: DevicePanelProps): ReactElement | undefined {
{device.revoked ? undefined : ( {device.revoked ? undefined : (
<Button <Button
data-testid={`device-revoke-${device.id}`} data-testid={`device-revoke-${device.id}`}
onClick={() => void revoke(device.id)} onClick={() => setConfirming(device)}
size="sm" size="sm"
> >
Revoke Revoke
@@ -203,6 +209,38 @@ export function DevicePanel(props: DevicePanelProps): ReactElement | undefined {
)} )}
</div> </div>
))} ))}
{/* Revoking strands the screen until someone re-pairs it in person, so the consequence is
spelled out before it happens — never a browser-native confirm. */}
<Dialog
actions={
<>
<Button
data-testid="device-revoke-cancel"
disabled={revoking}
onClick={dismissConfirmation}
>
Cancel
</Button>
<Button
data-testid="device-revoke-confirm"
disabled={revoking}
onClick={() => (confirming ? void revoke(confirming.id) : undefined)}
variant="danger"
>
Revoke display
</Button>
</>
}
onDismiss={dismissConfirmation}
open={confirming !== undefined}
title={`Revoke ${confirming?.name ?? "this display"}?`}
>
<p style={{ margin: 0 }}>
The screen stops showing content within a few seconds and returns to its pairing code.
This cannot be undone: to use it again, pair it from the screen with a new code.
</p>
</Dialog>
</section> </section>
); );
} }
+196 -18
View File
@@ -1,6 +1,6 @@
import { designToken } from "@ekklesia/design-tokens"; import { designToken } from "@ekklesia/design-tokens";
import type { ModuleUiImporter } from "@ekklesia/module-ui-host"; import type { ModuleUiImporter } from "@ekklesia/module-ui-host";
import { type ReactElement, useEffect, useState } from "react"; import { type ReactElement, useCallback, useEffect, useRef, useState } from "react";
import { import {
bootstrapDisplayHost, bootstrapDisplayHost,
type DisplayHostBootstrap, type DisplayHostBootstrap,
@@ -17,7 +17,13 @@ import { PairingScreen } from "./PairingScreen.js";
type DisplayState = type DisplayState =
| { readonly status: "loading"; readonly generation: number } | { readonly status: "loading"; readonly generation: number }
| { readonly status: "ready"; readonly bootstrap: DisplayHostBootstrap & { status: "ready" } } | {
readonly status: "ready";
readonly bootstrap: DisplayHostBootstrap & { status: "ready" };
/** Changes only when the mounted display must be replaced, never merely because the network
* came back. It keys the frame, so it is the one thing that can remount a running display. */
readonly generation: number;
}
/** No identity this display can use yet: show a pairing code rather than a human login form. */ /** No identity this display can use yet: show a pairing code rather than a human login form. */
| { readonly status: "pairing" } | { readonly status: "pairing" }
| { readonly status: "unavailable" } | { readonly status: "unavailable" }
@@ -31,6 +37,69 @@ export interface DisplayAppProps {
/** Where connectivity and cache-fallback signals come from (ADR 0043). The browser entry point /** Where connectivity and cache-fallback signals come from (ADR 0043). The browser entry point
* supplies the real ones; omitting it leaves a display that simply never reports being offline. */ * supplies the real ones; omitting it leaves a display that simply never reports being offline. */
readonly connectivity?: DisplayConnectivitySource; readonly connectivity?: DisplayConnectivitySource;
/** How long the display must stay offline before the corner indicator appears. Test seam only. */
readonly offlineIndicatorDelayMs?: number;
/** The shortest interval between two revalidations prompted by refused module requests. Test seam
* only. */
readonly revalidationIntervalMs?: number;
/** How often a running display re-resolves its descriptor on its own. Test seam only. */
readonly periodicRevalidationMs?: number;
/** The first automatic retry delay after a failed start; doubles to a one-minute ceiling. Test seam
* only. */
readonly retryBaseMs?: number;
}
/** A Wi-Fi reassociation or a switch reboot lasts seconds and needs no comment on a lobby wall. Only
* an outage that outlasts this is worth the corner indicator. */
export const DISPLAY_OFFLINE_INDICATOR_DELAY_MS = 10_000;
/** A module whose requests are being refused prompts at most one revalidation in this interval. A
* display that is merely missing one permission must not turn every refused request into two more. */
export const DISPLAY_REVALIDATION_INTERVAL_MS = 15_000;
/** A running display asks again this often, so a module upgraded on the server is picked up and a
* revocation is noticed even by a module that makes no requests of its own. */
export const DISPLAY_PERIODIC_REVALIDATION_MS = 5 * 60_000;
/** A display that could not start tries again on its own: from this delay, doubling to a minute. A
* kiosk that boots before its server — after a power cut, say — must come up without anyone pressing
* Retry. */
export const DISPLAY_RETRY_BASE_MS = 2_000;
export const DISPLAY_RETRY_MAX_MS = 60_000;
/** Whether a fresh resolution describes exactly the display already mounted. Compared structurally:
* the descriptor and principal are small, server-produced, and carry nothing volatile, so any
* difference — a new version, a new contribution, a different principal — is a real one. */
function sameDisplay(left: DisplayHostBootstrap, right: DisplayHostBootstrap): boolean {
return (
JSON.stringify([left.descriptor, left.principal]) ===
JSON.stringify([right.descriptor, right.principal])
);
}
/** `value`, but only once it has been continuously true for `delayMs`. Becoming false is immediate. */
function useSustained(value: boolean, delayMs: number): boolean {
const [sustained, setSustained] = useState(value && delayMs <= 0);
useEffect(() => {
if (!value) {
setSustained(false);
return;
}
if (delayMs <= 0) {
setSustained(true);
return;
}
const timer = setTimeout(() => setSustained(true), delayMs);
return () => clearTimeout(timer);
}, [delayMs, value]);
return sustained;
}
function isDenied(error: unknown): boolean {
return (
error instanceof HostProtocolError &&
(error.code === "DISPLAY_ACCESS_DENIED" || error.code === "HOST_UNAUTHENTICATED")
);
} }
const displayStyles = { const displayStyles = {
@@ -86,11 +155,13 @@ const displayStyles = {
* recovers on its own the moment connectivity returns. * recovers on its own the moment connectivity returns.
*/ */
function OfflineIndicator(props: { readonly connectivity: DisplayConnectivity }): ReactElement { function OfflineIndicator(props: { readonly connectivity: DisplayConnectivity }): ReactElement {
// Neutral wording: this is on a wall the congregation reads, and the product name is the
// installation's to choose, not the platform's (ADR 0046).
const label = props.connectivity.usingFallback const label = props.connectivity.usingFallback
? props.connectivity.fallbackKind === "artifact" ? props.connectivity.fallbackKind === "artifact"
? "Offline — showing the last downloaded display code" ? "Offline — showing the last downloaded display code"
: "Offline — showing the last downloaded Ekklesia display" : "Offline — showing the last downloaded display"
: "Offline — waiting for Ekklesia"; : "Offline — reconnecting";
return ( return (
<div data-testid="display-offline" role="status" style={displayStyles.offline}> <div data-testid="display-offline" role="status" style={displayStyles.offline}>
{label} {label}
@@ -130,7 +201,112 @@ export function DisplayApp(props: DisplayAppProps): ReactElement {
online: true, online: true,
usingFallback: false, usingFallback: false,
}); });
const { config, connectivity: connectivitySource, importModule, moduleId } = props; const {
config,
connectivity: connectivitySource,
importModule,
moduleId,
offlineIndicatorDelayMs = DISPLAY_OFFLINE_INDICATOR_DELAY_MS,
periodicRevalidationMs = DISPLAY_PERIODIC_REVALIDATION_MS,
retryBaseMs = DISPLAY_RETRY_BASE_MS,
revalidationIntervalMs = DISPLAY_REVALIDATION_INTERVAL_MS,
} = props;
/** Consecutive automatic retries since the display last started successfully. */
const failures = useRef(0);
const stateRef = useRef(state);
stateRef.current = state;
const revalidation = useRef<{ inFlight: AbortController | undefined; lastAt: number }>({
inFlight: undefined,
lastAt: 0,
});
/**
* Resolves the descriptor again **without** taking the running display down (ADR 0043).
*
* A display that is showing something keeps showing it while the question is asked. Only the
* answer decides what happens: the same display means nothing changes at all; a new version or a
* different principal replaces the frame; a refusal is a revoked or expired device and returns the
* screen to pairing (ADR 0036); and any other failure is the network still settling, so the frame
* stays. A display that is not showing anything simply starts over.
*/
const revalidate = useCallback(
(reason: "online" | "authorization" | "periodic"): void => {
const current = stateRef.current;
if (current.status !== "ready") {
if (reason === "online") {
setAttempt((value) => value + 1);
}
return;
}
const tracker = revalidation.current;
const nowMs = Date.now();
if (
tracker.inFlight !== undefined ||
(reason === "authorization" && nowMs - tracker.lastAt < revalidationIntervalMs)
) {
return;
}
tracker.lastAt = nowMs;
const abort = new AbortController();
tracker.inFlight = abort;
void bootstrapDisplayHost(config, moduleId, abort.signal)
.then((next) => {
const latest = stateRef.current;
if (abort.signal.aborted || latest.status !== "ready") {
return;
}
if (!sameDisplay(latest.bootstrap, next)) {
setState({ bootstrap: next, generation: latest.generation + 1, status: "ready" });
}
})
.catch((error: unknown) => {
if (!abort.signal.aborted && isDenied(error)) {
setState({ status: "pairing" });
}
})
.finally(() => {
if (tracker.inFlight === abort) {
tracker.inFlight = undefined;
}
});
},
[config, moduleId, revalidationIntervalMs],
);
useEffect(
() => () => {
revalidation.current.inFlight?.abort();
revalidation.current.inFlight = undefined;
},
[],
);
const onAuthorizationResponse = useCallback(() => revalidate("authorization"), [revalidate]);
// While running: ask again now and then, without disturbing anything when the answer is the same.
useEffect(() => {
if (state.status !== "ready" || periodicRevalidationMs <= 0) {
return;
}
const timer = setInterval(() => revalidate("periodic"), periodicRevalidationMs);
return () => clearInterval(timer);
}, [periodicRevalidationMs, revalidate, state.status]);
// Could not start: try again on its own, backing off. Pairing is not a failure — it has its own
// polling — and a ready display has nothing to retry.
useEffect(() => {
if (state.status === "ready") {
failures.current = 0;
return;
}
if (state.status !== "failed" && state.status !== "unavailable") {
return;
}
const delay = Math.min(DISPLAY_RETRY_MAX_MS, retryBaseMs * 2 ** Math.min(failures.current, 10));
failures.current += 1;
const timer = setTimeout(() => setAttempt((value) => value + 1), delay);
return () => clearTimeout(timer);
}, [retryBaseMs, state.status]);
useEffect(() => { useEffect(() => {
if (!connectivitySource) { if (!connectivitySource) {
@@ -142,27 +318,29 @@ export function DisplayApp(props: DisplayAppProps): ReactElement {
setConnectivity(next); setConnectivity(next);
// Nobody presses Retry on a lobby television. When the browser says the network is back, the // Nobody presses Retry on a lobby television. When the browser says the network is back, the
// display resolves its descriptor again — which is also what stops a screen from sitting on // display resolves its descriptor again — which is also what stops a screen from sitting on
// cached code after the server returns, since the fresh descriptor names the current version // cached code after the server returns, since a fresh descriptor naming a new version replaces
// and the network-first worker fetches it (ADR 0043). // the frame and the network-first worker fetches it (ADR 0043). The same answer leaves the
// running display exactly where it was.
if (next.online) { if (next.online) {
setAttempt((current) => current + 1); revalidate("online");
} }
}); });
return () => { return () => {
unsubscribe(); unsubscribe();
monitor.close(); monitor.close();
}; };
}, [connectivitySource]); }, [connectivitySource, revalidate]);
useEffect(() => { useEffect(() => {
const abort = new AbortController(); const abort = new AbortController();
revalidation.current.inFlight?.abort();
setState({ generation: attempt, status: "loading" }); setState({ generation: attempt, status: "loading" });
void bootstrapDisplayHost(config, moduleId, abort.signal) void bootstrapDisplayHost(config, moduleId, abort.signal)
.then((bootstrap) => { .then((bootstrap) => {
if (abort.signal.aborted) { if (abort.signal.aborted) {
return; return;
} }
setState({ bootstrap, status: "ready" }); setState({ bootstrap, generation: attempt, status: "ready" });
}) })
.catch((error: unknown) => { .catch((error: unknown) => {
if (abort.signal.aborted) { if (abort.signal.aborted) {
@@ -170,10 +348,7 @@ export function DisplayApp(props: DisplayAppProps): ReactElement {
} }
// A denied or unauthenticated descriptor is the ordinary state of a screen nobody has // A denied or unauthenticated descriptor is the ordinary state of a screen nobody has
// paired yet, not an error worth logging on every wall in the building. // paired yet, not an error worth logging on every wall in the building.
if ( if (isDenied(error)) {
error instanceof HostProtocolError &&
(error.code === "DISPLAY_ACCESS_DENIED" || error.code === "HOST_UNAUTHENTICATED")
) {
setState({ status: "pairing" }); setState({ status: "pairing" });
return; return;
} }
@@ -188,12 +363,14 @@ export function DisplayApp(props: DisplayAppProps): ReactElement {
}, [attempt, config, moduleId]); }, [attempt, config, moduleId]);
const retry = (): void => setAttempt((current) => current + 1); const retry = (): void => setAttempt((current) => current + 1);
const showOffline = useSustained(
!connectivity.online || connectivity.usingFallback,
offlineIndicatorDelayMs,
);
return ( return (
<div data-testid="display-host" style={displayStyles.host}> <div data-testid="display-host" style={displayStyles.host}>
{connectivity.online && !connectivity.usingFallback ? undefined : ( {showOffline ? <OfflineIndicator connectivity={connectivity} /> : undefined}
<OfflineIndicator connectivity={connectivity} />
)}
{state.status === "loading" ? ( {state.status === "loading" ? (
<DisplayMessage testId="display-host-loading">Loading display…</DisplayMessage> <DisplayMessage testId="display-host-loading">Loading display…</DisplayMessage>
) : undefined} ) : undefined}
@@ -215,7 +392,8 @@ export function DisplayApp(props: DisplayAppProps): ReactElement {
config={config} config={config}
descriptor={state.bootstrap.descriptor} descriptor={state.bootstrap.descriptor}
{...(importModule ? { importModule } : {})} {...(importModule ? { importModule } : {})}
key={`${state.bootstrap.descriptor.id}@${state.bootstrap.descriptor.version}:${attempt}`} key={`${state.bootstrap.descriptor.id}@${state.bootstrap.descriptor.version}:${state.generation}`}
onAuthorizationResponse={onAuthorizationResponse}
principal={state.bootstrap.principal} principal={state.bootstrap.principal}
/> />
) : undefined} ) : undefined}
+12 -1
View File
@@ -35,6 +35,9 @@ export interface DisplayUiFrameProps {
readonly principal: ModuleUiPrincipal; readonly principal: ModuleUiPrincipal;
/** Test/composition seam only. Production uses the generic loader's native dynamic import. */ /** Test/composition seam only. Production uses the generic loader's native dynamic import. */
readonly importModule?: ModuleUiImporter; readonly importModule?: ModuleUiImporter;
/** Host-internal signal that a module request was refused, so the host can find out whether this
* device was revoked. It never enters the module-facing context. */
readonly onAuthorizationResponse?: (status: 401 | 403) => void;
} }
interface DisplayUiAttemptProps extends DisplayUiFrameProps { interface DisplayUiAttemptProps extends DisplayUiFrameProps {
@@ -94,7 +97,10 @@ const frameStyles = {
function DisplayUiAttempt(props: DisplayUiAttemptProps): ReactElement { function DisplayUiAttempt(props: DisplayUiAttemptProps): ReactElement {
const containerRef = useRef<HTMLDivElement | null>(null); const containerRef = useRef<HTMLDivElement | null>(null);
const [status, setStatus] = useState<FrameStatus>("loading"); const [status, setStatus] = useState<FrameStatus>("loading");
const { config, descriptor, importModule, onRetry, principal } = props; const { config, descriptor, importModule, onAuthorizationResponse, onRetry, principal } = props;
// Read through a ref so a new callback identity can never remount a running display.
const authorizationRef = useRef(onAuthorizationResponse);
authorizationRef.current = onAuthorizationResponse;
useEffect(() => { useEffect(() => {
const container = containerRef.current; const container = containerRef.current;
@@ -162,6 +168,11 @@ function DisplayUiAttempt(props: DisplayUiAttemptProps): ReactElement {
context: createModuleUiContext({ context: createModuleUiContext({
config, config,
descriptor, descriptor,
onAuthorizationResponse: (status) => {
if (!cancelled) {
authorizationRef.current?.(status);
}
},
principal, principal,
...(importModule ? { importModule } : {}), ...(importModule ? { importModule } : {}),
}), }),
+106
View File
@@ -0,0 +1,106 @@
import { act } from "react";
import { createRoot, type Root } from "react-dom/client";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { DevicePanel } from "../src/browser/DevicePanel.js";
declare global {
// eslint-disable-next-line no-var
var IS_REACT_ACT_ENVIRONMENT: boolean | undefined;
}
const apiBaseUrl = "https://ekklesia.example.test";
let container: HTMLDivElement;
let root: Root;
beforeEach(() => {
globalThis.IS_REACT_ACT_ENVIRONMENT = true;
container = document.createElement("div");
document.body.append(container);
root = createRoot(container);
});
afterEach(async () => {
await act(async () => root.unmount());
container.remove();
});
function json(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
headers: { "content-type": "application/json" },
status,
});
}
function server() {
let revoked = false;
const revokes: string[] = [];
const fetch = vi.fn(async (input: RequestInfo | URL) => {
const url = String(input);
if (url.endsWith("/api/v1/devices")) {
return json({
devices: [{ id: "device-1", name: "Foyer TV", permissions: [], revoked }],
});
}
if (url.endsWith("/api/v1/devices/device-1/revoke")) {
revoked = true;
revokes.push(url);
return new Response(null, { status: 204 });
}
return json({ code: "NOT_FOUND" }, 404);
}) as unknown as typeof globalThis.fetch;
return { fetch, revokes };
}
function query(testId: string): HTMLElement | null {
return container.querySelector(`[data-testid="${testId}"]`);
}
async function click(testId: string): Promise<void> {
await act(async () => {
query(testId)?.click();
for (let index = 0; index < 5; index += 1) {
await new Promise((resolve) => setTimeout(resolve, 0));
}
});
}
/**
* Revoking a display strands that screen until someone re-pairs it in person, so the panel explains
* the consequence in its own dialog first — never with a browser-native confirm (DESIGN §21).
*/
describe("display device revocation", () => {
it("asks in a product dialog before revoking, and does nothing on cancel", async () => {
const platform = server();
const confirm = vi.fn(() => true);
Object.assign(window, { confirm });
await act(async () => {
root.render(<DevicePanel config={{ apiBaseUrl, fetch: platform.fetch }} />);
});
await click("device-revoke-device-1");
const dialog = container.querySelector('[role="dialog"]');
expect(dialog?.textContent).toContain("Revoke Foyer TV?");
expect(dialog?.textContent).toContain("pairing code");
expect(platform.revokes).toEqual([]);
await click("device-revoke-cancel");
expect(container.querySelector('[role="dialog"]')).toBeNull();
expect(platform.revokes).toEqual([]);
expect(confirm).not.toHaveBeenCalled();
});
it("revokes only after the destructive action is confirmed", async () => {
const platform = server();
await act(async () => {
root.render(<DevicePanel config={{ apiBaseUrl, fetch: platform.fetch }} />);
});
await click("device-revoke-device-1");
await click("device-revoke-confirm");
expect(platform.revokes).toHaveLength(1);
expect(container.querySelector('[role="dialog"]')).toBeNull();
expect(query("device-device-1")?.textContent).toContain("(revoked)");
});
});
+301
View File
@@ -180,6 +180,12 @@ async function render(
importModule?: (url: string) => Promise<unknown>, importModule?: (url: string) => Promise<unknown>,
moduleId = "announcements", moduleId = "announcements",
connectivity?: DisplayConnectivitySource, connectivity?: DisplayConnectivitySource,
timing: {
readonly offlineIndicatorDelayMs?: number;
readonly revalidationIntervalMs?: number;
} = {
offlineIndicatorDelayMs: 0,
},
): Promise<void> { ): Promise<void> {
await act(async () => { await act(async () => {
root.render( root.render(
@@ -187,6 +193,7 @@ async function render(
config={config} config={config}
{...(connectivity ? { connectivity } : {})} {...(connectivity ? { connectivity } : {})}
{...(importModule ? { importModule } : {})} {...(importModule ? { importModule } : {})}
{...timing}
moduleId={moduleId} moduleId={moduleId}
/>, />,
); );
@@ -926,3 +933,297 @@ describe("display host — offline behaviour", () => {
expect(query("display-pairing")).not.toBeNull(); expect(query("display-pairing")).not.toBeNull();
}); });
}); });
/**
* Stage 0 regressions: a running display survives its network, and a revoked one returns to pairing.
*
* Before these, every browser `online` event tore the module down and mounted it again (a fresh
* viewer session, a fresh stage, every cached object URL released), and a device revoked while its
* display was open kept playing until someone reloaded the page.
*/
describe("display host — continuity across reconnects and revocation", () => {
function controllableConnectivity() {
const listeners = new Map<string, Set<() => void>>();
return {
source: {
addEventListener(type: "online" | "offline", listener: () => void) {
const existing = listeners.get(type) ?? new Set();
existing.add(listener);
listeners.set(type, existing);
},
initialOnline: true,
removeEventListener(type: "online" | "offline", listener: () => void) {
listeners.get(type)?.delete(listener);
},
} satisfies DisplayConnectivitySource,
async emit(type: "online" | "offline") {
await act(async () => {
for (const listener of listeners.get(type) ?? []) {
listener();
}
});
},
};
}
async function flush(): Promise<void> {
await act(async () => {
for (let index = 0; index < 5; index += 1) {
await new Promise((resolve) => setTimeout(resolve, 0));
}
});
}
/** A server whose descriptor answer can be changed mid-test, counting descriptor requests. */
function changingServer() {
let descriptor: { status: number; body: unknown } = { body: descriptorPayload(), status: 200 };
let descriptorCalls = 0;
const fetch = vi.fn(async (input: RequestInfo | URL) => {
const url = String(input);
if (url.endsWith("/api/v1/module-displays/announcements")) {
descriptorCalls += 1;
return json(descriptor.body, descriptor.status);
}
if (url.endsWith("/api/v1/device-pairing/requests")) {
return json(
{ code: "482731", expiresAt: new Date().toISOString(), pollIntervalMs: 60_000 },
201,
);
}
if (url.endsWith("/api/v1/device-pairing/claims")) {
return json({ status: "pending" });
}
if (url.endsWith("/api/v1/modules/announcements/refused")) {
return json({ code: "AUTHORIZATION_DENIED" }, 403);
}
return json({ code: "NOT_FOUND" }, 404);
}) as unknown as typeof globalThis.fetch;
return {
descriptorCalls: () => descriptorCalls,
fetch,
set(status: number, body: unknown) {
descriptor = { body, status };
},
};
}
function countingArtifact(version = "1.2.0") {
const counts = { mounts: 0, unmounts: 0 };
let context: ModuleUiHostContext | undefined;
const importModule = async (url: string) =>
artifact("announcements", url.includes("/2.0.0/") ? "2.0.0" : version, {
mount(element, mounted) {
counts.mounts += 1;
context = mounted;
element.textContent = "live";
return () => {
counts.unmounts += 1;
};
},
});
return { context: () => context, counts, importModule };
}
it("does not remount a live display when the network drops and returns", async () => {
const server = changingServer();
const connectivity = controllableConnectivity();
const display = countingArtifact();
await render(
{ apiBaseUrl, fetch: server.fetch },
display.importModule,
"announcements",
connectivity.source,
);
await flush();
expect(display.counts.mounts).toBe(1);
const before = server.descriptorCalls();
await connectivity.emit("offline");
await connectivity.emit("online");
await flush();
// The descriptor is still resolved again (ADR 0043) — but the same answer changes nothing.
expect(server.descriptorCalls()).toBeGreaterThan(before);
expect(display.counts).toEqual({ mounts: 1, unmounts: 0 });
expect(query("display-host-loading")).toBeNull();
});
it("keeps the running display when the descriptor cannot be reached yet", async () => {
const server = changingServer();
const connectivity = controllableConnectivity();
const display = countingArtifact();
await render(
{ apiBaseUrl, fetch: server.fetch },
display.importModule,
"announcements",
connectivity.source,
);
await flush();
server.set(503, { code: "UNAVAILABLE" });
await connectivity.emit("offline");
await connectivity.emit("online");
await flush();
expect(display.counts).toEqual({ mounts: 1, unmounts: 0 });
expect(query("display-host-error")).toBeNull();
});
it("replaces the display when the server now names a new version", async () => {
const server = changingServer();
const connectivity = controllableConnectivity();
const display = countingArtifact();
await render(
{ apiBaseUrl, fetch: server.fetch },
display.importModule,
"announcements",
connectivity.source,
);
await flush();
server.set(200, descriptorPayload("announcements", "2.0.0"));
await connectivity.emit("offline");
await connectivity.emit("online");
await flush();
expect(display.counts.unmounts).toBe(1);
expect(display.counts.mounts).toBe(2);
});
it("returns a device revoked while its display is open to pairing, without a reload", async () => {
const server = changingServer();
const display = countingArtifact();
await render({ apiBaseUrl, fetch: server.fetch }, display.importModule, "announcements");
await flush();
expect(query("display-module-container")).not.toBeNull();
// The administrator revokes the device. Its next module request is refused.
server.set(403, { code: "AUTHORIZATION_DENIED" });
await act(async () => {
await display.context()?.api.request("refused");
});
await flush();
expect(display.counts.unmounts).toBe(1);
expect(query("display-pairing")).not.toBeNull();
});
it("does not remount when a refused request turns out not to be a revocation", async () => {
const server = changingServer();
const display = countingArtifact();
await render(
{ apiBaseUrl, fetch: server.fetch },
display.importModule,
"announcements",
undefined,
{
offlineIndicatorDelayMs: 0,
revalidationIntervalMs: 60_000,
},
);
await flush();
const before = server.descriptorCalls();
await act(async () => {
await display.context()?.api.request("refused");
await display.context()?.api.request("refused");
await display.context()?.api.request("refused");
});
await flush();
// One revalidation for the burst, and the display stays exactly where it was.
expect(server.descriptorCalls()).toBe(before + 1);
expect(display.counts).toEqual({ mounts: 1, unmounts: 0 });
});
it("does not show the offline indicator for a momentary blip", async () => {
const server = changingServer();
const connectivity = controllableConnectivity();
await render(
{ apiBaseUrl, fetch: server.fetch },
countingArtifact().importModule,
"announcements",
connectivity.source,
{ offlineIndicatorDelayMs: 60 },
);
await flush();
await connectivity.emit("offline");
await connectivity.emit("online");
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 120));
});
expect(query("display-offline")).toBeNull();
await connectivity.emit("offline");
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 120));
});
expect(query("display-offline")?.textContent).toBe("Offline — reconnecting");
});
it("starts on its own when the server comes up after the screen did", async () => {
// A kiosk that boots before its server: the network is up, so no `online` event will ever come.
let up = false;
const fetch = vi.fn(async (input: RequestInfo | URL) => {
const url = String(input);
if (url.endsWith("/api/v1/module-displays/announcements")) {
return up ? json(descriptorPayload()) : json({ code: "UNAVAILABLE" }, 503);
}
return json({ code: "NOT_FOUND" }, 404);
}) as unknown as typeof globalThis.fetch;
const display = countingArtifact();
await act(async () => {
root.render(
<DisplayApp
config={{ apiBaseUrl, fetch }}
importModule={display.importModule}
moduleId="announcements"
offlineIndicatorDelayMs={0}
retryBaseMs={20}
/>,
);
});
await flush();
expect(query("display-host-error")).not.toBeNull();
up = true;
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 200));
});
await flush();
expect(display.counts.mounts).toBe(1);
expect(query("display-host-error")).toBeNull();
});
it("re-resolves a running display periodically, remounting only for a new version", async () => {
const server = changingServer();
const display = countingArtifact();
await act(async () => {
root.render(
<DisplayApp
config={{ apiBaseUrl, fetch: server.fetch }}
importModule={display.importModule}
moduleId="announcements"
offlineIndicatorDelayMs={0}
periodicRevalidationMs={40}
/>,
);
});
await flush();
const before = server.descriptorCalls();
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 150));
});
await flush();
expect(server.descriptorCalls()).toBeGreaterThan(before);
expect(display.counts).toEqual({ mounts: 1, unmounts: 0 });
server.set(200, descriptorPayload("announcements", "2.0.0"));
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 150));
});
await flush();
expect(display.counts.mounts).toBe(2);
});
});
+55 -6
View File
@@ -157,6 +157,21 @@ describe("CI scripts on a shared Docker daemon", () => {
} }
}); });
it("exercise a real native package through the hardened verifier container", async () => {
const imageCheck = (await source("scripts/check-image.sh")).replace(/\\\n\s*/gu, " ");
expect(imageCheck).toContain(
"packages/module-pack/tests/support/pack-native-verifier-proof.mjs",
);
expect(imageCheck).toContain("--tmpfs /tmp:rw,nosuid,nodev,noexec,size=64m");
expect(imageCheck).toContain(
"--tmpfs /run/ekklesia-verifier:rw,nosuid,nodev,exec,size=1100m,mode=0700,uid=1000,gid=1000",
);
expect(imageCheck).toContain("const actual=await m.proof()");
expect(imageCheck).toContain("file:///tmp/noexec-installed/server/index.js");
expect(imageCheck).toContain("scratch_failure /tmp");
expect(imageCheck).toContain("scratch_failure /package");
});
it("install the smoke test's module only as a canonical package the image itself verified (ADR 0049)", async () => { it("install the smoke test's module only as a canonical package the image itself verified (ADR 0049)", async () => {
const deploy = (await source("scripts/smoke-deploy.sh")).replace(/\\\n\s*/gu, " "); const deploy = (await source("scripts/smoke-deploy.sh")).replace(/\\\n\s*/gu, " ");
const packaging = await source("scripts/lib/build-module-package.Dockerfile"); const packaging = await source("scripts/lib/build-module-package.Dockerfile");
@@ -164,19 +179,28 @@ describe("CI scripts on a shared Docker daemon", () => {
expect(packaging).not.toMatch(/pnpm[^\n]*\bdeploy\b/u); expect(packaging).not.toMatch(/pnpm[^\n]*\bdeploy\b/u);
expect(deploy).toContain("scripts/lib/build-module-package.Dockerfile"); expect(deploy).toContain("scripts/lib/build-module-package.Dockerfile");
expect(deploy).toMatch( expect(deploy).toMatch(
/--entrypoint node "\$image" core\/dist\/cli\/verify-module-package\.js[^\n]*--sha256-file[^\n]*--extract-to \/staging\/example/u, /--entrypoint node "\$image" core\/dist\/cli\/verify-module-package\.js[^\n]*--sha256-file[^\n]*--extract-to "\/staging\/\$\{module_id\}"/u,
); );
expect(deploy).toContain("--tmpfs /tmp:rw,nosuid,nodev,noexec,size=64m");
expect(deploy).toContain(
"--tmpfs /run/ekklesia-verifier:rw,nosuid,nodev,exec,size=1100m,mode=0700,uid=1000,gid=1000",
);
expect(deploy).toContain("-e EKKLESIA_MODULE_VERIFIER_SCRATCH_DIR=/run/ekklesia-verifier");
// Module storage only ever receives the verified tree from staging. // Module storage only ever receives the verified tree from staging.
expect(deploy).toContain("cp -R /staging/example /modules/.incoming-example"); expect(deploy).toContain(`staged="/staging/\${MODULE_ID}"`);
expect(deploy).toContain('cp -R "$staged" "$incoming"');
expect(deploy).toContain('mv "$incoming" "$installed"');
}); });
it("hand the verifier the package's archive and digest together, from a fresh release pair, never from build output", async () => { it("hand the verifier the package's archive and digest together, from a fresh release pair, never from build output", async () => {
const deploy = (await source("scripts/smoke-deploy.sh")).replace(/\\\n\s*/gu, " "); const deploy = (await source("scripts/smoke-deploy.sh")).replace(/\\\n\s*/gu, " ");
expect(deploy).toContain('source "$repository_root/scripts/lib/module-package-release.sh"'); expect(deploy).toContain('source "$repository_root/scripts/lib/module-package-release.sh"');
// The pair is made from this run's own build output, into a directory that must not exist yet. // The pair is made from this run's own build output, into a directory that must not exist yet.
expect(deploy).toContain('--output "type=local,dest=${work}/module-package"'); expect(deploy).toContain(`--output "type=local,dest=\${work}/module-package"`);
expect(deploy).toContain('package_output="$work/module-package"');
expect(deploy).toContain('package_output="$module_release_directory"');
expect(deploy).toContain( expect(deploy).toContain(
'package_archive="$(module_package_release "$work/module-package" "$work/package")"', 'package_archive="$(module_package_release "$package_output" "$work/package")"',
); );
expect(deploy).toMatch(/^\s*stage package$/mu); expect(deploy).toMatch(/^\s*stage package$/mu);
// BuildKit's output directory is 0700 and owned by whoever built it: it is never mounted. // BuildKit's output directory is 0700 and owned by whoever built it: it is never mounted.
@@ -187,11 +211,36 @@ describe("CI scripts on a shared Docker daemon", () => {
.split("\n") .split("\n")
.find((line) => line.includes("core/dist/cli/verify-module-package.js")); .find((line) => line.includes("core/dist/cli/verify-module-package.js"));
expect(verifier).toContain('-v "$daemon_work/package:/package:ro"'); expect(verifier).toContain('-v "$daemon_work/package:/package:ro"');
expect(verifier).toContain('"/package/${package_archive}"'); expect(verifier).toContain(`"/package/\${package_archive}"`);
expect(verifier).toContain('--sha256-file "/package/${package_archive}.sha256"'); expect(verifier).toContain(`--sha256-file "/package/\${package_archive}.sha256"`);
expect(verifier).not.toMatch(/--sha256\s/u); expect(verifier).not.toMatch(/--sha256\s/u);
}); });
it("accepts one exact external module release without putting its source in the platform image", async () => {
const deploy = await source("scripts/smoke-deploy.sh");
expect(deploy).toContain(`module_release_directory="\${EKKLESIA_SMOKE_MODULE_RELEASE_DIR:-}"`);
expect(deploy).toContain(`module_id="\${EKKLESIA_SMOKE_MODULE_ID:-example}"`);
expect(deploy).toContain('package_output="$module_release_directory"');
expect(deploy).toContain('-e SMOKE_MODULE_ID="$module_id"');
expect(deploy).toContain(`--extract-to "/staging/\${module_id}"`);
expect(deploy).toContain(`installed="/modules/\${MODULE_ID}"`);
});
it("proves installed module state across restart, container recreation, and lifecycle changes", async () => {
const deploy = await source("scripts/smoke-deploy.sh");
expect(deploy).toContain(`"\${compose[@]}" rm --force api`);
expect(deploy).toContain("container recreation made an applied module migration pending again");
expect(deploy).toContain('core/dist/cli/set-module-enabled.js --disable --module "$module_id"');
expect(deploy).toContain('core/dist/cli/set-module-enabled.js --enable --module "$module_id"');
expect(deploy).toContain("the restored display runtime did not return with its data");
});
it("gives page-side display requests enough time on a constrained shared runner", async () => {
const deploy = await source("scripts/smoke-deploy.sh");
expect(deploy).toContain('-e SMOKE_CALL_TIMEOUT_MS="');
expect(deploy).toContain("EKKLESIA_SMOKE_CALL_TIMEOUT_MS:-30000");
});
it("run every deploy smoke against the one generic image, never a second image with a module in it", async () => { it("run every deploy smoke against the one generic image, never a second image with a module in it", async () => {
// ADR 0048: the split topology installs its test module into module storage at run time. // ADR 0048: the split topology installs its test module into module storage at run time.
const deploy = (await source("scripts/smoke-deploy.sh")).replace(/\\\n\s*/gu, " "); const deploy = (await source("scripts/smoke-deploy.sh")).replace(/\\\n\s*/gu, " ");
+1 -1
View File
@@ -138,7 +138,7 @@ describe("reference deployment compose files", () => {
]), ]),
); );
expect(services["object-storage"]?.command).toEqual( expect(services["object-storage"]?.command).toEqual(
expect.arrayContaining(["-s3.port.https=8443"]), expect.arrayContaining(["-master.volumeSizeLimitMB=32", "-s3.port.https=8443"]),
); );
const hba = (await text("postgres/pg_hba.conf")) const hba = (await text("postgres/pg_hba.conf"))
+8 -8
View File
@@ -13,18 +13,18 @@
"typecheck": "tsc --noEmit --project tsconfig.json && tsc --noEmit --project tsconfig.ui.json" "typecheck": "tsc --noEmit --project tsconfig.json && tsc --noEmit --project tsconfig.ui.json"
}, },
"dependencies": { "dependencies": {
"@ekklesia/design-tokens": "^0.2.1", "@ekklesia/design-tokens": "^0.2.4",
"@ekklesia/module-database": "^0.2.1", "@ekklesia/module-database": "^0.2.4",
"@ekklesia/module-sdk": "^0.2.1", "@ekklesia/module-sdk": "^0.2.4",
"@ekklesia/module-ui-sdk": "^0.2.1", "@ekklesia/module-ui-sdk": "^0.2.4",
"@ekklesia/ui": "^0.2.1", "@ekklesia/ui": "^0.2.4",
"react": "19.2.8", "react": "19.2.8",
"react-dom": "19.2.8" "react-dom": "19.2.8"
}, },
"devDependencies": { "devDependencies": {
"@ekklesia/module-pack": "0.2.1", "@ekklesia/module-pack": "0.2.4",
"@ekklesia/module-ui-build": "^0.2.1", "@ekklesia/module-ui-build": "^0.2.4",
"@ekklesia/typescript-config": "^0.2.1", "@ekklesia/typescript-config": "^0.2.4",
"@types/node": "24.13.3", "@types/node": "24.13.3",
"@types/react": "19.2.18", "@types/react": "19.2.18",
"@types/react-dom": "19.2.4", "@types/react-dom": "19.2.4",