Merge pull request 'feat: the public @ekklesia package release train, built, reproduced, and proven (ADR 0053)' (#80) from feat/public-package-release-train into main
CI / verify (push) Successful in 1m5s
CI / infrastructure-smoke (push) Successful in 37s
CI / migration-smoke (push) Successful in 1m54s
CI / package-release (push) Successful in 32s
CI / image-build (push) Successful in 10s
CI / deploy-smoke (same-origin) (push) Successful in 49s
CI / deploy-smoke (split) (push) Successful in 1m22s

Reviewed-on: https://git.riverofglorymn.org/River_of_Glory_Church/Ekklesia/pulls/80
Reviewed-by: The_River <[email protected]>
This commit was merged in pull request #80.
This commit is contained in:
The_River
2026-10-08 13:53:41 +00:00
17 changed files with 1316 additions and 370 deletions
+8 -5
View File
@@ -128,7 +128,7 @@ jobs:
if: always() if: always()
run: bash scripts/ci-sweep.sh --job run: bash scripts/ci-sweep.sh --job
standalone-module: package-release:
needs: verify needs: verify
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 15 timeout-minutes: 15
@@ -154,10 +154,13 @@ jobs:
- name: Install dependencies - name: Install dependencies
run: pnpm install --frozen-lockfile run: pnpm install --frozen-lockfile
# ADR 0053: pack and inspect the public packages, then install them into a module outside this # ADR 0053's package release gate: build the public package train from this commit, build it
# repository and prove it builds, packages, and verifies. Nothing is published. # again from a second checkout and require byte-identical artifacts, then install exactly those
- name: Build a standalone module from the packed public packages # tarballs (offline, no registry) into a module outside this repository and prove it builds,
run: pnpm test:standalone-module # packages, and verifies. Nothing is published and no credential exists here. A future publish
# job must depend on this job and publish only the artifacts it proved.
- name: Build, reproduce, and consume the public package train
run: pnpm test:package-release
image-build: image-build:
needs: verify needs: verify
+6 -2
View File
@@ -1,4 +1,8 @@
node_modules/ node_modules/
# pnpm's content-addressed package store. pnpm keeps it beside node_modules on the same file system,
# so when the checkout is its own mount (a CI runner's workspace volume, a bind-mounted container),
# `pnpm install` creates it here, at the checkout's root. A package cache, never source.
.pnpm-store/
.turbo/ .turbo/
coverage/ coverage/
dist/ dist/
@@ -20,5 +24,5 @@ core/.identity-schema-check-*
# The packager tests copy fixture modules here (packages/module-pack/tests/support/fixture-module.ts). # The packager tests copy fixture modules here (packages/module-pack/tests/support/fixture-module.ts).
packages/module-pack/.fixtures/ packages/module-pack/.fixtures/
# `pnpm packages:pack` writes the packed public packages here (scripts/pack-public-packages.ts). # `pnpm packages:release` writes package release trains here (scripts/package-release.ts).
.public-packages/ .package-release/
+3
View File
@@ -78,6 +78,9 @@ ever disagree:
- `pnpm test:migrations:integration` — live, disposable-PostgreSQL evidence: migration discovery, - `pnpm test:migrations:integration` — live, disposable-PostgreSQL evidence: migration discovery,
planning, and execution, plus the `apps/api` HTTP transport (signup, sign-in, principal planning, and execution, plus the `apps/api` HTTP transport (signup, sign-in, principal
resolution, sign-out). Requires Docker. Run this for any change touching either area. resolution, sign-out). Requires Docker. Run this for any change touching either area.
- `pnpm test:package-release` — builds the public `@ekklesia/*` package train, proves it
reproducible, and proves a standalone module consumes exactly those tarballs. Publishes nothing.
Run it for any change to the public packages or their release tooling.
- `pnpm doctor` — read-only local readiness report. Never treat it as a substitute for `pnpm - `pnpm doctor` — read-only local readiness report. Never treat it as a substitute for `pnpm
verify` or the integration suite. verify` or the integration suite.
- `pnpm migrations:plan` / `pnpm migrations:apply` — read-only planning vs. mutating execution - `pnpm migrations:plan` / `pnpm migrations:apply` — read-only planning vs. mutating execution
+6 -4
View File
@@ -63,11 +63,13 @@ omits licence material, the packager may add reviewed licence supplements
operating-system or CPU targets, signing, and any installer or marketplace do not exist yet. operating-system or CPU targets, signing, and any installer or marketplace do not exist yet.
A module can be built in its own repository from Ekklesia's public module-development packages A module can be built in its own repository from Ekklesia's public module-development packages
([ADR 0053](docs/architecture/decisions/0053-standalone-module-development-packages.md), Proposed): ([ADR 0053](docs/architecture/decisions/0053-standalone-module-development-packages.md)):
a curated `@ekklesia/module-sdk`, `@ekklesia/module-database`, the UI packages, and the build and a curated `@ekklesia/module-sdk`, `@ekklesia/module-database`, the UI packages, and the build and
packaging tools. CI packs and inspects those packages and proves a module outside the repository packaging tools. They are released as one coordinated train of tarballs with a release manifest,
installs, builds, packages, and verifies from the tarballs alone. They are not published to any and CI proves on every run that two builds of a commit are byte-identical and that a module outside
registry yet. the repository installs, builds, packages, and verifies from those artifacts alone. They are not
published to any registry yet
([Releases and the public package train](docs/development/releases.md)).
Phase 4A added Core-owned identity foundations: a Core-owned Better Auth database schema Phase 4A added Core-owned identity foundations: a Core-owned Better Auth database schema
(`ekklesia_core_user`, `ekklesia_core_session`, `ekklesia_core_account`, (`ekklesia_core_user`, `ekklesia_core_session`, `ekklesia_core_account`,
@@ -1,6 +1,6 @@
# 0053: A module builds outside the monorepo, from Ekklesia packages published over the npm registry protocol # 0053: A module builds outside the monorepo, from Ekklesia packages published over the npm registry protocol
- Status: Proposed - Status: Accepted
- Date: 2026-10-07 - Date: 2026-10-07
- Extends: [0049](0049-module-package-v1.md). This is the step ADR 0049 deferred: "publishing the - Extends: [0049](0049-module-package-v1.md). This is the step ADR 0049 deferred: "publishing the
Ekklesia SDK packages for module builds outside a platform workspace". A registry is used only to Ekklesia SDK packages for module builds outside a platform workspace". A registry is used only to
+1 -1
View File
@@ -61,4 +61,4 @@ ADRs capture durable choices that affect more than one package or phase.
- [0050: Module Database ABI v1: modules and Core exchange SQL, never ORM objects](0050-module-database-abi-v1.md) — Accepted - [0050: Module Database ABI v1: modules and Core exchange SQL, never ORM objects](0050-module-database-abi-v1.md) — Accepted
- [0051: Module packages declare their own license and carry its text](0051-module-package-license-material.md) — Accepted - [0051: Module packages declare their own license and carry its text](0051-module-package-license-material.md) — Accepted
- [0052: Reviewed license supplements cover redistributed dependencies whose packages omit required material](0052-reviewed-license-supplements.md) — Proposed - [0052: Reviewed license supplements cover redistributed dependencies whose packages omit required material](0052-reviewed-license-supplements.md) — Proposed
- [0053: A module builds outside the monorepo, from Ekklesia packages published over the npm registry protocol](0053-standalone-module-development-packages.md) — Proposed - [0053: A module builds outside the monorepo, from Ekklesia packages published over the npm registry protocol](0053-standalone-module-development-packages.md) — Accepted
+194
View File
@@ -0,0 +1,194 @@
# Releases and the public package train
An Ekklesia platform release is one version. The platform image, the private workspace packages,
and the public `@ekklesia/*` module-development packages all carry it. This page covers the order a
release happens in and the public package train
([ADR 0053](../architecture/decisions/0053-standalone-module-development-packages.md)): how its
artifacts are produced, what proves them, and what a future publication step will do with them.
> **Registry publication is not implemented.** Nothing in this repository publishes a package, and
> no CI job holds a registry credential. The release artifacts below are produced and proven on
> every CI run. A future publication step publishes **exactly these artifacts**, unchanged, and
> nothing else.
## Where each step sits
```text
version set ─▶ build ─▶ test ─▶ pack the public package train ─▶ standalone consumer proof
│
tag and release (maintainer) ◀─ [future: publish] ◀──┘
```
1. **Version set.** A `chore(release): prepare X` pull request sets the release version: the root
`package.json`, every platform workspace package (the module template versions as a module, not
with the platform), the image build example in `deploy/README.md`, and the standalone fixture's
exact `@ekklesia/module-pack` pin (`tests/fixtures/standalone-module/package.json`). The tag
must point at a commit that carries the version it names.
2. **Build and test.** `pnpm verify`, the integration suite, and the deploy smoke, as for any change.
3. **Pack the public package train.** `pnpm packages:release` (below).
4. **Standalone consumer proof.** A module outside the repository consumes exactly those
artifacts (below).
5. **[Future] publish.** Not implemented. Once it is, it must run only after steps 3 and 4 passed
for the same commit. It must check every artifact against the release manifest before
publishing it, and publish nothing else.
6. **Tag and release.** Unchanged: the maintainer tags the merge commit of the version-set pull
request. Nothing here creates, moves, or requires a tag.
Steps 2 to 4 run in CI on every pull request and on `main`. The `package-release` job is the gate a
future publication job must depend on.
## The public package train
| Package | Role |
| --- | --- |
| `@ekklesia/module-sdk`, `@ekklesia/module-database`, `@ekklesia/module-ui-sdk`, `@ekklesia/design-tokens` | author API |
| `@ekklesia/module-ui-build`, `@ekklesia/module-pack`, `@ekklesia/typescript-config` | tooling |
| `@ekklesia/ui` | publishable, pre-1.0 component API |
| `@ekklesia/contracts` | implementation dependency of the packages above: published only because they need it, never a supported direct dependency of a module |
The list lives in `scripts/package-release.ts` (`PUBLIC_PACKAGES`). Every other workspace package is
`private`, and the release builder refuses to run if a package outside the list is publishable or
carries the `module-development` distribution marker.
- **One version.** Every package in the train has the platform release version.
- **One train.** Every `@ekklesia/*` dependency in a packed manifest names a package of the same
train at exactly that version. `workspace:` ranges become those exact versions when packed.
- **No independent versions.** A package cannot be released on its own.
## Producing the release artifacts
```bash
pnpm packages:release # writes .package-release/ekklesia-packages-<version>/
```
`scripts/package-release.ts`:
1. **Refuses a modified source tree, before it builds anything.** A release is built from a
commit. A tree is clean when `git status` shows no change to a tracked file and no untracked file
that Git does not ignore. Ignored files are build output and package-manager state:
`node_modules`, `dist`, `.turbo`, and pnpm's `.pnpm-store`, which pnpm creates at the checkout's
root when the checkout is its own mount (a CI runner's workspace volume). The failure lists every
change. `--allow-modified` exists for local development only; its manifest then says
`"modified": true`, and such a release must never be published.
2. **Checks the release set and versions** in the workspace.
3. **Rebuilds from clean output.** It removes each public package's `dist/` and rebuilds without the
build cache, so output from a deleted source file can never ship. It then checks the source tree
again: a build that changed a tracked file, or wrote an untracked one Git does not ignore, fails
the release, because what is packed must be exactly the commit.
4. **Packs each package with `pnpm pack`.** Pack-only pnpm hooks (`scripts/lib/pack-hooks.cjs`)
apply, never the workspace's install settings ([Reproducibility](#reproducibility)).
5. **Inspects every tarball.** It checks:
- the expected name, version, and file name;
- a `LICENSE`, and only the `files` the manifest declares, each built file with the source it
came from, and no source, test, or configuration file;
- no `workspace:` range, no `devDependencies`, no `publishConfig`, and no install-time lifecycle
script;
- only public `@ekklesia/*` dependencies, at the train's version;
- the distribution marker;
- no local path (the checkout, the home directory, or the temporary directory) in any file.
6. **Checks the set.** The release must hold exactly the expected tarballs: none missing, none extra.
7. **Writes the release directory only when everything passed,** in one step. It refuses to
overwrite one that exists.
```text
ekklesia-packages-<version>/
├── ekklesia-contracts-<version>.tgz
├── ekklesia-design-tokens-<version>.tgz
├── … one .tgz per package in the train
├── SHA256SUMS `sha256sum -c SHA256SUMS` compatible
└── ekklesia-package-train.json the release manifest
```
### The release manifest
`ekklesia-package-train.json` describes the train by content. It is provider-neutral: it names no
registry and is not a registry catalog.
```json
{
"packageTrain": 1,
"version": "0.1.2",
"source": { "modified": false, "revision": "<full commit id>" },
"packages": [
{
"file": "ekklesia-contracts-0.1.2.tgz",
"integrity": "sha512-<base64>",
"name": "@ekklesia/contracts",
"role": "implementation",
"sha256": "<hex>",
"size": 87379,
"version": "0.1.2"
}
]
}
```
`integrity` is the npm `dist.integrity` form: a registry computes the same value from the same
bytes, and a module's lockfile records it. The packages are sorted by name, and the file's bytes
depend only on the commit.
## Proving the artifacts
```bash
pnpm test:package-release # scripts/test-package-release.sh
```
1. **Builds the train** from this checkout. Nothing is built before it.
2. **Builds it again** from a second worktree of the same commit at another absolute path, and
requires both release directories to be byte-identical: every tarball, `SHA256SUMS`, and the
manifest.
3. **Builds Core's verifier**, then **runs the standalone consumer proof** (`scripts/test-standalone-module.sh --release <dir>`) on
that release:
- **The release is checked first.** Every artifact must match the manifest's size, SHA-256, and
integrity, and the release must not be built from a modified tree.
- **The fixture module installs from those tarballs alone.** It is
`tests/fixtures/standalone-module`, copied outside the repository. Every `@ekklesia/*`
package, direct or transitive, resolves to its release tarball. There is no workspace link and
no Ekklesia source checkout. The install runs offline: third-party packages come from the
workspace install's own pnpm store, named explicitly because pnpm otherwise picks a store per
file system, so no registry is involved at all.
- **The lockfile matches the release.** It must record each artifact's own integrity. The module
then reinstalls from the lockfile alone with `--frozen-lockfile`.
- **The install has the right shape.** There is no `drizzle-orm`, and exactly one `react`, which
`@ekklesia/ui` resolves to as well. `@ekklesia/contracts` is installed only transitively: it is
not in the module's `package.json` and not directly under `node_modules/@ekklesia`. Every
installed `@ekklesia/*` package, `module-pack` included, is exactly the release version, and
the fixture pins `module-pack` exactly.
- **The module builds.** It typechecks with `skipLibCheck: false`, so every published declaration
it reaches is checked, then builds its backend and UI.
- **It packages and verifies.** It is packed with the release's own `ekklesia-pack-module`, with
no workspace file present. Core's verifier then verifies it. The package must:
- incorporate the train's packages at the release version;
- name the release's module-pack as its packager;
- credit no Ekklesia package and exactly one React;
- contain no local path.
## Reproducibility
Two builds of the same commit produce byte-identical tarballs, and CI proves it on every run.
- **pnpm normalizes the tar entries.** Every entry gets the fixed modification time 1985-10-26, owner
0, and a mode of 0644, or 0755 when executable. The gzip header records no time.
- **The build output is deterministic,** and source maps use relative paths.
- **One nondeterminism came from pnpm itself.** pnpm rewrites `workspace:` ranges concurrently and
inserts each dependency as its lookup resolves. A package with more than one `workspace:`
dependency in one field could therefore be packed with those keys in a different order. Builds
from two checkouts of one commit differed in exactly that way: `@ekklesia/module-pack`'s
`package.json` ordered `@ekklesia/module-sdk` and `@ekklesia/module-database` differently.
- **The fix is pnpm's own hook.** The pack-only `beforePacking` hook puts each dependency map back
in the source `package.json`'s key order. It changes no name, range, or field. The tarball stays
an ordinary `pnpm pack` tarball.
- **The inspection enforces it.** It fails any packed map whose order differs from its source.
- **The hook also drops `devDependencies`** from packed manifests. A consumer never installs them,
and a tooling package's test-only development dependencies named private workspace packages
outside the train.
**What reproducibility does not cover:**
- The module package the consumer proof builds is deterministic only for a fixed install layout.
pnpm names a `file:` tarball's install directory after its path, and bundlers keep those names in
comments. So the proof always installs from the same relative path. From a registry, the directory
is simply `<name>@<version>`.
- Byte identity is proven for the pinned toolchain: Node.js from `.node-version`, and pnpm from
`packageManager`. Another Node.js or zlib build may compress differently. Each artifact's
integrity is recorded either way, and that, not byte identity, is what a publication step must
verify.
+11
View File
@@ -649,6 +649,17 @@ Ekklesia Module Package v1 (ADR 0049) is proven at three levels, all in `pnpm ve
builds the template and Core first, and runs the template's own tests (which rebuild its UI in builds the template and Core first, and runs the template's own tests (which rebuild its UI in
place) before these. place) before these.
## The public package train
`pnpm test:package-release` (the CI `package-release` job) builds the public `@ekklesia/*` package
train, proves a second build from another checkout of the commit byte-identical, and installs
exactly those tarballs into the standalone fixture module, offline, which must typecheck with
`skipLibCheck: false`, build, package, and verify. It needs a clean source tree; pass
`-- --allow-modified` to try uncommitted changes locally. The release rules themselves (set,
versions, manifest inspection, release directory, manifest bytes, the pack hook) are unit-tested in
`tests/public-packages.test.ts`, part of `pnpm verify`. See
[Releases and the public package train](releases.md).
## The generic platform image and the reference deployment ## The generic platform image and the reference deployment
These need Docker and build images locally; nothing is pushed or published. These need Docker and build images locally; nothing is pushed or published.
+5 -4
View File
@@ -157,10 +157,11 @@ Packaging and verification work exactly as for a module in this repository
their identity, not by where they are installed, so they are recorded under `incorporates` and never their identity, not by where they are installed, so they are recorded under `incorporates` and never
credited as third-party software, wherever they came from. credited as third-party software, wherever they came from.
Ekklesia proves this before any release: `pnpm packages:pack` builds, packs, and inspects the public Ekklesia proves this on every CI run: `pnpm test:package-release` builds the public packages of a
packages, and `bash scripts/test-standalone-module.sh` installs them into a module outside the release as one train of tarballs, proves two builds identical, and installs exactly those tarballs
repository (`tests/fixtures/standalone-module`), with no workspace link and no Ekklesia registry, into a module outside the repository (`tests/fixtures/standalone-module`), with no workspace link
then typechecks, builds, packages, and verifies it. and no registry, then typechecks, builds, packages, and verifies it. Those artifacts are what a
future publication step publishes ([Releases and the public package train](../development/releases.md)).
### An installed module is self-contained ### An installed module is self-contained
+2 -2
View File
@@ -38,11 +38,11 @@
"modules:pack": "turbo run build && node packages/module-pack/bin/ekklesia-pack-module.mjs", "modules:pack": "turbo run build && node packages/module-pack/bin/ekklesia-pack-module.mjs",
"modules:validate": "turbo run build && node core/dist/cli/validate-modules.js", "modules:validate": "turbo run build && node core/dist/cli/validate-modules.js",
"modules:verify-package": "turbo run build && node core/dist/cli/verify-module-package.js", "modules:verify-package": "turbo run build && node core/dist/cli/verify-module-package.js",
"packages:pack": "node scripts/pack-public-packages.ts --out .public-packages", "packages:release": "node scripts/package-release.ts --out .package-release",
"test": "turbo run test test:root", "test": "turbo run test test:root",
"test:migrations:integration": "pnpm build && bash scripts/test-migrations-integration.sh", "test:migrations:integration": "pnpm build && bash scripts/test-migrations-integration.sh",
"test:root": "vitest run --config vitest.config.ts", "test:root": "vitest run --config vitest.config.ts",
"test:standalone-module": "pnpm build && bash scripts/test-standalone-module.sh", "test:package-release": "bash scripts/test-package-release.sh",
"test:watch": "vitest", "test:watch": "vitest",
"typecheck": "turbo run typecheck typecheck:root", "typecheck": "turbo run typecheck typecheck:root",
"typecheck:root": "tsc --noEmit --project tsconfig.json", "typecheck:root": "tsc --noEmit --project tsconfig.json",
+56
View File
@@ -0,0 +1,56 @@
// pnpm hooks for packing the public package train, and nothing else: passed to `pnpm pack` with
// `--pnpmfile` by scripts/package-release.ts, never installed as the workspace's .pnpmfile.cjs, so
// installs and the lockfile are unaffected.
//
// `beforePacking` runs after pnpm has rewritten `workspace:` ranges into exact versions, and does two
// things to the packed package.json only:
//
// - It drops `devDependencies`. A consumer never installs them, and a tooling package's test-only
// development dependencies name private workspace packages (Core, the module template) that are
// not part of the public package train.
// - It puts every remaining dependency map back in the source package.json's own key order. pnpm
// rewrites `workspace:` ranges concurrently and inserts each entry as its lookup resolves, so two
// packs of the same commit could order those keys differently, the one difference that made
// otherwise byte-identical tarballs differ. No name or range changes.
"use strict";
const { readFileSync } = require("node:fs");
const path = require("node:path");
const DEPENDENCY_FIELDS = [
"dependencies",
"optionalDependencies",
"peerDependencies",
"peerDependenciesMeta",
];
function inSourceOrder(packed, source) {
if (
packed === null ||
typeof packed !== "object" ||
source === null ||
typeof source !== "object"
) {
return packed;
}
const ordered = {};
for (const key of Object.keys(source)) {
if (Object.hasOwn(packed, key)) ordered[key] = packed[key];
}
for (const key of Object.keys(packed)) {
if (!Object.hasOwn(ordered, key)) ordered[key] = packed[key];
}
return ordered;
}
function beforePacking(manifest, directory) {
const source = JSON.parse(readFileSync(path.join(directory, "package.json"), "utf8"));
const packed = { ...manifest };
delete packed.devDependencies;
for (const field of DEPENDENCY_FIELDS) {
if (packed[field] !== undefined) packed[field] = inSourceOrder(packed[field], source[field]);
}
return packed;
}
module.exports = { hooks: { beforePacking }, inSourceOrder };
+8
View File
@@ -0,0 +1,8 @@
/** Types for `pack-hooks.cjs`, the pnpm hooks applied to `pnpm pack` by scripts/package-release.ts. */
type Manifest = Record<string, unknown>;
export declare const hooks: {
beforePacking(manifest: Manifest, directory: string): Manifest;
};
export declare function inSourceOrder(packed: unknown, source: unknown): unknown;
-304
View File
@@ -1,304 +0,0 @@
import { execFile } from "node:child_process";
import { mkdir, mkdtemp, readdir, readFile, realpath, rm, stat } from "node:fs/promises";
import { homedir, tmpdir } from "node:os";
import path from "node:path";
import process from "node:process";
import { fileURLToPath, pathToFileURL } from "node:url";
import { promisify } from "node:util";
/**
* The public module-development package set (ADR 0053), and the release check that packs it.
*
* node scripts/pack-public-packages.ts --out <directory>
*
* Rebuilds each public package from clean output, packs it with `pnpm pack`, and inspects every
* tarball before anything may consume it: only the package's built files, its declarations, and a
* `LICENSE`; a manifest with no `workspace:` range, no install-time lifecycle script, the explicit
* distribution marker, and the one coordinated version. It publishes nothing.
*/
const execFileAsync = promisify(execFile);
const repositoryRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
export type PublicPackageRole = "author-api" | "tooling" | "pre-1.0" | "implementation";
export interface PublicPackage {
readonly name: string;
/** Relative to the repository root. */
readonly directory: string;
readonly role: PublicPackageRole;
}
/** ADR 0053's public set. Every other workspace package is private and never published. */
export const PUBLIC_PACKAGES: readonly PublicPackage[] = Object.freeze([
{ directory: "packages/module-sdk", name: "@ekklesia/module-sdk", role: "author-api" },
{ directory: "packages/module-database", name: "@ekklesia/module-database", role: "author-api" },
{ directory: "packages/module-ui-sdk", name: "@ekklesia/module-ui-sdk", role: "author-api" },
{ directory: "packages/design-tokens", name: "@ekklesia/design-tokens", role: "author-api" },
{ directory: "packages/module-ui-build", name: "@ekklesia/module-ui-build", role: "tooling" },
{ directory: "packages/module-pack", name: "@ekklesia/module-pack", role: "tooling" },
{
directory: "packages/typescript-config",
name: "@ekklesia/typescript-config",
role: "tooling",
},
{ directory: "packages/ui", name: "@ekklesia/ui", role: "pre-1.0" },
// A dependency of the packages above, never a supported direct module-author dependency.
{ directory: "packages/contracts", name: "@ekklesia/contracts", role: "implementation" },
]);
/** `ekklesia.distribution` in a public package's manifest: the identity the canonical packager
* recognizes an installed Ekklesia package by (`packages/module-pack/src/packages.ts`). */
export const PUBLIC_PACKAGE_DISTRIBUTION = "module-development";
/** Scripts a package manager runs on a consumer's machine when it installs, packs, or publishes. */
const LIFECYCLE_SCRIPTS = Object.freeze([
"preinstall",
"install",
"postinstall",
"prepare",
"prepack",
"postpack",
"prepublish",
"prepublishOnly",
"publish",
"postpublish",
]);
/** The fields a consumer's install resolves. `devDependencies` are never installed from a published
* package, so a tooling package's test-only use of a private package is not a problem there. */
const INSTALLED_DEPENDENCY_FIELDS = Object.freeze([
"dependencies",
"peerDependencies",
"optionalDependencies",
]);
function isRecord(value: unknown): value is Readonly<Record<string, unknown>> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
/** What is wrong with a packed `package.json`, or nothing. */
export function packedManifestProblems(
manifest: Readonly<Record<string, unknown>>,
expected: { readonly name: string; readonly version: string },
): string[] {
const problems: string[] = [];
if (manifest.name !== expected.name) problems.push(`is named ${String(manifest.name)}`);
if (manifest.version !== expected.version) {
problems.push(
`has version ${String(manifest.version)}, not the coordinated ${expected.version}`,
);
}
if (manifest.private !== undefined) problems.push("declares private");
if (manifest.license !== "AGPL-3.0-or-later") problems.push("does not declare its licence");
if (JSON.stringify(manifest).includes("workspace:")) problems.push("keeps a workspace: range");
const ekklesia = manifest.ekklesia;
if (!isRecord(ekklesia) || ekklesia.distribution !== PUBLIC_PACKAGE_DISTRIBUTION) {
problems.push("lacks the module-development distribution marker");
}
const scripts = isRecord(manifest.scripts) ? manifest.scripts : {};
for (const script of LIFECYCLE_SCRIPTS) {
if (Object.hasOwn(scripts, script)) problems.push(`runs a ${script} script`);
}
const publicNames = new Set(PUBLIC_PACKAGES.map((entry) => entry.name));
for (const field of INSTALLED_DEPENDENCY_FIELDS) {
const dependencies = manifest[field];
if (!isRecord(dependencies)) continue;
for (const [name, range] of Object.entries(dependencies)) {
if (!name.startsWith("@ekklesia/")) continue;
if (!publicNames.has(name)) problems.push(`${field} names the private ${name}`);
else if (range !== expected.version) {
problems.push(`${field} names ${name}@${String(range)}, not ${expected.version}`);
}
}
}
return problems;
}
const BUILT_FILE = /^dist\/(.+?)(?:\.d\.ts|\.js)(?:\.map)?$/u;
/**
* What is wrong with a tarball's file list, or nothing. Every file is `package.json`, `LICENSE`, or
* under one of the manifest's `files` entries; nothing is a source, test, or configuration file; and
* every built file has the source it was compiled from, so stale output from a deleted source fails.
*/
export function tarballFileProblems(
files: readonly string[],
manifest: Readonly<Record<string, unknown>>,
hasSource: (relativeWithoutExtension: string) => boolean,
): string[] {
const problems: string[] = [];
const declared = Array.isArray(manifest.files) ? manifest.files.map(String) : [];
if (!files.includes("LICENSE")) problems.push("has no LICENSE");
if (!files.includes("package.json")) problems.push("has no package.json");
for (const file of files) {
if (file === "package.json" || file === "LICENSE") continue;
if (!declared.some((entry) => file === entry || file.startsWith(`${entry}/`))) {
problems.push(`ships ${file}, which its files field does not name`);
continue;
}
if (
/(?:^|\/)(?:src|tests?|node_modules)\//u.test(file) ||
/\.test\.[cm]?[jt]sx?$/u.test(file)
) {
problems.push(`ships ${file}`);
continue;
}
const built = BUILT_FILE.exec(file);
if (file.startsWith("dist/") && (built === null || !hasSource(built[1] ?? ""))) {
problems.push(`ships ${file}, which no source produces`);
}
}
return problems;
}
async function readJson(file: string): Promise<Record<string, unknown>> {
return JSON.parse(await readFile(file, "utf8")) as Record<string, unknown>;
}
async function filesUnder(directory: string): Promise<string[]> {
const entries = await readdir(directory, { recursive: true, withFileTypes: true });
return entries
.filter((entry) => entry.isFile())
.map((entry) =>
path.relative(directory, path.join(entry.parentPath, entry.name)).split(path.sep).join("/"),
)
.sort();
}
async function sourceExists(packageDirectory: string, relative: string): Promise<boolean> {
for (const extension of [".ts", ".tsx"]) {
try {
await stat(path.join(packageDirectory, "src", `${relative}${extension}`));
return true;
} catch {}
}
return false;
}
export interface PackedPackage {
readonly name: string;
readonly tarball: string;
readonly files: number;
readonly bytes: number;
}
async function pnpm(arguments_: readonly string[], cwd: string): Promise<string> {
const { stdout } = await execFileAsync("pnpm", [...arguments_], {
cwd,
maxBuffer: 64 * 1024 * 1024,
});
return stdout;
}
async function packOne(
entry: PublicPackage,
version: string,
outDirectory: string,
scratch: string,
): Promise<{ packed: PackedPackage; problems: string[] }> {
const directory = path.join(repositoryRoot, entry.directory);
const report = JSON.parse(
await pnpm(["pack", "--json", "--pack-destination", outDirectory], directory),
) as { filename: string };
const tarball = report.filename;
const extracted = path.join(scratch, entry.name.replace("/", "+"));
await mkdir(extracted, { recursive: true });
await execFileAsync("tar", ["-xzf", tarball, "-C", extracted]);
const root = path.join(extracted, "package");
const files = await filesUnder(root);
const manifest = await readJson(path.join(root, "package.json"));
const sources = new Map<string, boolean>();
for (const file of files) {
const built = BUILT_FILE.exec(file)?.[1];
if (built !== undefined && !sources.has(built)) {
sources.set(built, await sourceExists(directory, built));
}
}
const problems = [
...packedManifestProblems(manifest, { name: entry.name, version }),
...tarballFileProblems(files, manifest, (relative) => sources.get(relative) === true),
];
// Nothing the package says may name the machine or checkout it was packed on.
for (const file of files) {
const text = (await readFile(path.join(root, file))).toString("latin1");
for (const local of [repositoryRoot, homedir()]) {
if (text.includes(local)) problems.push(`${file} names a local path`);
}
}
return {
packed: { bytes: (await stat(tarball)).size, files: files.length, name: entry.name, tarball },
problems: [...new Set(problems)],
};
}
/** Rebuilds the public set from clean output, packs it into `outDirectory`, and inspects it. */
export async function packPublicPackages(outDirectory: string): Promise<{
readonly packed: readonly PackedPackage[];
readonly problems: readonly string[];
}> {
const version = String((await readJson(path.join(repositoryRoot, "package.json"))).version);
await mkdir(outDirectory, { recursive: true });
const out = await realpath(outDirectory);
// Stale output from a deleted source would otherwise ship: the build only ever adds files, and
// Turbo's cache would restore whatever an earlier build left behind.
for (const entry of PUBLIC_PACKAGES) {
await rm(path.join(repositoryRoot, entry.directory, "dist"), { force: true, recursive: true });
}
await pnpm(
[
"exec",
"turbo",
"run",
"build",
"--force",
"--output-logs=errors-only",
...PUBLIC_PACKAGES.map((entry) => `--filter=${entry.name}`),
],
repositoryRoot,
);
const scratch = await mkdtemp(path.join(tmpdir(), "ekklesia-public-packages-"));
try {
const packed: PackedPackage[] = [];
const problems: string[] = [];
for (const entry of PUBLIC_PACKAGES) {
const result = await packOne(entry, version, out, scratch);
packed.push(result.packed);
problems.push(...result.problems.map((problem) => `${entry.name} ${problem}.`));
}
return { packed, problems };
} finally {
await rm(scratch, { force: true, recursive: true });
}
}
async function main(argv: readonly string[]): Promise<number> {
if (argv.length !== 2 || argv[0] !== "--out" || !argv[1]) {
process.stderr.write("Usage: node scripts/pack-public-packages.ts --out <directory>\n");
return 64;
}
try {
const { packed, problems } = await packPublicPackages(path.resolve(argv[1]));
for (const entry of packed) {
process.stdout.write(
`${path.basename(entry.tarball)}: ${entry.files} files, ${entry.bytes} bytes\n`,
);
}
if (problems.length > 0) {
for (const problem of problems) process.stderr.write(`${problem}\n`);
return 1;
}
process.stdout.write(`Packed and inspected ${packed.length} public packages.\n`);
return 0;
} catch {
process.stderr.write("Packing the public packages failed while building or packing.\n");
return 1;
}
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
process.exitCode = await main(process.argv.slice(2));
}
+622
View File
@@ -0,0 +1,622 @@
import { execFile } from "node:child_process";
import { createHash } from "node:crypto";
import { mkdir, mkdtemp, readdir, readFile, rename, rm, stat, writeFile } from "node:fs/promises";
import { homedir, tmpdir } from "node:os";
import path from "node:path";
import process from "node:process";
import { fileURLToPath, pathToFileURL } from "node:url";
import { promisify } from "node:util";
/**
* The public package release train (ADR 0053): the canonical way to produce the exact
* `@ekklesia/*` package tarballs of one platform release. It publishes nothing.
*
* node scripts/package-release.ts --out <directory> [--allow-modified]
*
* From a clean source tree, it removes the public packages' build output, rebuilds them without the
* build cache, packs each with `pnpm pack`, inspects every tarball, and only when everything passed
* writes the release directory `<directory>/ekklesia-packages-<version>/`:
*
* ekklesia-<name>-<version>.tgz one per public package, exactly the expected set
* SHA256SUMS `sha256sum -c` compatible
* ekklesia-package-train.json the release manifest (`packageTrainManifest`)
*
* Those files are what a future registry publication step publishes, unchanged, after checking them
* against the manifest. `--allow-modified` builds from a source tree with uncommitted changes, for
* local development only; the manifest then says so, and such a release must never be published.
*/
const execFileAsync = promisify(execFile);
const repositoryRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
/** pnpm hooks applied to `pnpm pack` only (see the file itself). */
const PACK_HOOKS = path.join(repositoryRoot, "scripts", "lib", "pack-hooks.cjs");
export type PublicPackageRole = "author-api" | "tooling" | "pre-1.0" | "implementation";
export interface PublicPackage {
readonly name: string;
/** Relative to the repository root. */
readonly directory: string;
readonly role: PublicPackageRole;
}
/** ADR 0053's public set. Every other workspace package is private and never published. */
export const PUBLIC_PACKAGES: readonly PublicPackage[] = Object.freeze([
{ directory: "packages/module-sdk", name: "@ekklesia/module-sdk", role: "author-api" },
{ directory: "packages/module-database", name: "@ekklesia/module-database", role: "author-api" },
{ directory: "packages/module-ui-sdk", name: "@ekklesia/module-ui-sdk", role: "author-api" },
{ directory: "packages/design-tokens", name: "@ekklesia/design-tokens", role: "author-api" },
{ directory: "packages/module-ui-build", name: "@ekklesia/module-ui-build", role: "tooling" },
{ directory: "packages/module-pack", name: "@ekklesia/module-pack", role: "tooling" },
{
directory: "packages/typescript-config",
name: "@ekklesia/typescript-config",
role: "tooling",
},
{ directory: "packages/ui", name: "@ekklesia/ui", role: "pre-1.0" },
// A dependency of the packages above, never a supported direct module-author dependency.
{ directory: "packages/contracts", name: "@ekklesia/contracts", role: "implementation" },
]);
/** `ekklesia.distribution` in a public package's manifest: the identity the canonical packager
* recognizes an installed Ekklesia package by (`packages/module-pack/src/packages.ts`). */
export const PUBLIC_PACKAGE_DISTRIBUTION = "module-development";
/** The release manifest's file name, and its format version. */
export const PACKAGE_TRAIN_MANIFEST = "ekklesia-package-train.json";
export const PACKAGE_TRAIN_FORMAT = 1;
/** `@ekklesia/module-sdk` → `ekklesia-module-sdk-<version>.tgz`, the name `pnpm pack` writes. */
export function tarballName(name: string, version: string): string {
return `${name.replace(/^@/u, "").replace("/", "-")}-${version}.tgz`;
}
/** The directory one release's artifacts are written to, inside `--out`. */
export function releaseDirectoryName(version: string): string {
return `ekklesia-packages-${version}`;
}
/** Scripts a package manager runs on a consumer's machine when it installs, packs, or publishes. */
const LIFECYCLE_SCRIPTS = Object.freeze([
"preinstall",
"install",
"postinstall",
"prepare",
"prepack",
"postpack",
"prepublish",
"prepublishOnly",
"publish",
"postpublish",
]);
const DEPENDENCY_FIELDS = Object.freeze([
"dependencies",
"peerDependencies",
"optionalDependencies",
"devDependencies",
]);
function isRecord(value: unknown): value is Readonly<Record<string, unknown>> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
/**
* What is wrong with a packed `package.json`, or nothing. `source` is the package's own source
* manifest, when there is one to compare with: a packed dependency map must keep its key order.
*/
export function packedManifestProblems(
manifest: Readonly<Record<string, unknown>>,
expected: { readonly name: string; readonly version: string },
source?: Readonly<Record<string, unknown>>,
): string[] {
const problems: string[] = [];
if (manifest.name !== expected.name) problems.push(`is named ${String(manifest.name)}`);
if (manifest.version !== expected.version) {
problems.push(
`has version ${String(manifest.version)}, not the coordinated ${expected.version}`,
);
}
if (manifest.private !== undefined) problems.push("declares private");
if (manifest.license !== "AGPL-3.0-or-later") problems.push("does not declare its licence");
if (JSON.stringify(manifest).includes("workspace:")) problems.push("keeps a workspace: range");
// A registry belongs to the publication step's configuration, never to an artifact.
if (manifest.publishConfig !== undefined) problems.push("names a publishConfig");
if (manifest.devDependencies !== undefined) problems.push("keeps devDependencies");
const ekklesia = manifest.ekklesia;
if (!isRecord(ekklesia) || ekklesia.distribution !== PUBLIC_PACKAGE_DISTRIBUTION) {
problems.push("lacks the module-development distribution marker");
}
const scripts = isRecord(manifest.scripts) ? manifest.scripts : {};
for (const script of LIFECYCLE_SCRIPTS) {
if (Object.hasOwn(scripts, script)) problems.push(`runs a ${script} script`);
}
const publicNames = new Set(PUBLIC_PACKAGES.map((entry) => entry.name));
for (const field of DEPENDENCY_FIELDS) {
const dependencies = manifest[field];
if (!isRecord(dependencies)) continue;
for (const [name, range] of Object.entries(dependencies)) {
if (!name.startsWith("@ekklesia/")) continue;
if (!publicNames.has(name)) problems.push(`${field} names the private ${name}`);
else if (range !== expected.version) {
problems.push(`${field} names ${name}@${String(range)}, not ${expected.version}`);
}
}
const sourceOrder = source?.[field];
if (isRecord(sourceOrder)) {
const order = Object.keys(sourceOrder).filter((key) => Object.hasOwn(dependencies, key));
if (JSON.stringify(order) !== JSON.stringify(Object.keys(dependencies))) {
problems.push(`orders ${field} differently from its source`);
}
}
}
return problems;
}
const BUILT_FILE = /^dist\/(.+?)(?:\.d\.ts|\.js)(?:\.map)?$/u;
/**
* What is wrong with a tarball's file list, or nothing. Every file is `package.json`, `LICENSE`, or
* under one of the manifest's `files` entries; nothing is a source, test, or configuration file; and
* every built file has the source it was compiled from, so stale output from a deleted source fails.
*/
export function tarballFileProblems(
files: readonly string[],
manifest: Readonly<Record<string, unknown>>,
hasSource: (relativeWithoutExtension: string) => boolean,
): string[] {
const problems: string[] = [];
const declared = Array.isArray(manifest.files) ? manifest.files.map(String) : [];
if (!files.includes("LICENSE")) problems.push("has no LICENSE");
if (!files.includes("package.json")) problems.push("has no package.json");
for (const file of files) {
if (file === "package.json" || file === "LICENSE") continue;
if (!declared.some((entry) => file === entry || file.startsWith(`${entry}/`))) {
problems.push(`ships ${file}, which its files field does not name`);
continue;
}
if (
/(?:^|\/)(?:src|tests?|node_modules)\//u.test(file) ||
/\.test\.[cm]?[jt]sx?$/u.test(file)
) {
problems.push(`ships ${file}`);
continue;
}
const built = BUILT_FILE.exec(file);
if (file.startsWith("dist/") && (built === null || !hasSource(built[1] ?? ""))) {
problems.push(`ships ${file}, which no source produces`);
}
}
return problems;
}
export interface WorkspacePackage {
readonly name: string;
/** Relative to the repository root. */
readonly directory: string;
readonly manifest: Readonly<Record<string, unknown>>;
}
/** Every package `pnpm-workspace.yaml` names, read without a YAML dependency: the file lists
* literal directories and `<directory>/*` patterns only. */
export async function workspacePackages(root = repositoryRoot): Promise<WorkspacePackage[]> {
const workspace = await readFile(path.join(root, "pnpm-workspace.yaml"), "utf8");
const block = /^packages:\n((?:[ \t]+-[^\n]*\n)+)/mu.exec(workspace)?.[1] ?? "";
const patterns = [...block.matchAll(/-\s*["']?([^"'\n]+?)["']?\s*$/gmu)].map(
(match) => match[1] ?? "",
);
const packages: WorkspacePackage[] = [];
for (const pattern of patterns) {
const directories = pattern.endsWith("/*")
? (
await readdir(path.join(root, pattern.slice(0, -2)), { withFileTypes: true }).catch(
() => [],
)
)
.filter((entry) => entry.isDirectory())
.map((entry) => `${pattern.slice(0, -2)}/${entry.name}`)
: [pattern];
for (const directory of directories) {
const manifest = await readJson(path.join(root, directory, "package.json")).catch(
() => undefined,
);
if (manifest && typeof manifest.name === "string") {
packages.push({ directory, manifest, name: manifest.name });
}
}
}
return packages.sort((left, right) => (left.name < right.name ? -1 : 1));
}
/**
* What is wrong with the workspace's release set, or nothing: every public package exists where the
* list says, carries the marker and the platform version, and is not private; every other package is
* private and unmarked, so nothing can enter the train unnoticed.
*/
export function releaseSetProblems(
packages: readonly WorkspacePackage[],
platformVersion: string,
): string[] {
const problems: string[] = [];
const byName = new Map(packages.map((entry) => [entry.name, entry]));
const publicNames = new Set(PUBLIC_PACKAGES.map((entry) => entry.name));
for (const expected of PUBLIC_PACKAGES) {
const found = byName.get(expected.name);
if (found === undefined || found.directory !== expected.directory) {
problems.push(`${expected.name} is missing from ${expected.directory}.`);
continue;
}
if (found.manifest.version !== platformVersion) {
problems.push(
`${expected.name} is version ${String(found.manifest.version)}, not the platform's ${platformVersion}.`,
);
}
if (found.manifest.private !== undefined) problems.push(`${expected.name} is private.`);
}
for (const entry of packages) {
if (publicNames.has(entry.name)) continue;
const ekklesia = entry.manifest.ekklesia;
if (isRecord(ekklesia) && ekklesia.distribution === PUBLIC_PACKAGE_DISTRIBUTION) {
problems.push(`${entry.name} carries the distribution marker but is not in the release set.`);
}
if (entry.manifest.private !== true) {
problems.push(`${entry.name} is publishable but not in the release set.`);
}
}
return problems;
}
export interface TrainPackage {
readonly name: string;
readonly version: string;
readonly role: PublicPackageRole;
readonly file: string;
readonly size: number;
/** Hexadecimal SHA-256 of the tarball. */
readonly sha256: string;
/** The npm `dist.integrity` of the tarball: `sha512-<base64>`. */
readonly integrity: string;
}
/** The release manifest: what one train is, by content. Provider-neutral, and no registry catalog. */
export interface PackageTrainManifest {
readonly packageTrain: typeof PACKAGE_TRAIN_FORMAT;
/** The platform release version, and every package's version. */
readonly version: string;
readonly source: {
readonly revision: string;
/** True only for a local `--allow-modified` build, which must never be published. */
readonly modified: boolean;
};
readonly packages: readonly TrainPackage[];
}
/** What is wrong with a release directory's contents against its manifest, or nothing. */
export function releaseDirectoryProblems(
manifest: PackageTrainManifest,
files: ReadonlyMap<string, Uint8Array>,
): string[] {
const problems: string[] = [];
if (manifest.packageTrain !== PACKAGE_TRAIN_FORMAT) problems.push("unknown manifest format.");
const expected = new Map(
PUBLIC_PACKAGES.map((entry) => [tarballName(entry.name, manifest.version), entry.name]),
);
const listed = new Map(manifest.packages.map((entry) => [entry.file, entry]));
for (const [file, name] of expected) {
if (!listed.has(file)) problems.push(`the manifest does not list ${name}.`);
if (!files.has(file)) problems.push(`${file} is missing.`);
}
for (const file of files.keys()) {
if (!expected.has(file) && file !== PACKAGE_TRAIN_MANIFEST && file !== "SHA256SUMS") {
problems.push(`${file} is not part of the release.`);
}
}
for (const entry of manifest.packages) {
if (!expected.has(entry.file) || expected.get(entry.file) !== entry.name) {
problems.push(`the manifest lists ${entry.name} as ${entry.file}, which is not expected.`);
continue;
}
if (entry.version !== manifest.version) {
problems.push(`the manifest lists ${entry.name} at ${entry.version}.`);
}
const data = files.get(entry.file);
if (data === undefined) continue;
const digests = artifactDigests(data);
if (
data.length !== entry.size ||
digests.sha256 !== entry.sha256 ||
digests.integrity !== entry.integrity
) {
problems.push(`${entry.file} does not match its size, SHA-256, or integrity.`);
}
}
return problems;
}
export function artifactDigests(data: Uint8Array): { sha256: string; integrity: string } {
return {
integrity: `sha512-${createHash("sha512").update(data).digest("base64")}`,
sha256: createHash("sha256").update(data).digest("hex"),
};
}
/** The manifest's exact bytes: sorted packages, two-space JSON, one trailing newline. */
export function serializeManifest(manifest: PackageTrainManifest): string {
const packages = [...manifest.packages]
.sort((left, right) => (left.name < right.name ? -1 : 1))
.map((entry) => ({
file: entry.file,
integrity: entry.integrity,
name: entry.name,
role: entry.role,
sha256: entry.sha256,
size: entry.size,
version: entry.version,
}));
return `${JSON.stringify(
{
packageTrain: manifest.packageTrain,
version: manifest.version,
source: { modified: manifest.source.modified, revision: manifest.source.revision },
packages,
},
null,
2,
)}\n`;
}
async function readJson(file: string): Promise<Record<string, unknown>> {
return JSON.parse(await readFile(file, "utf8")) as Record<string, unknown>;
}
async function filesUnder(directory: string): Promise<string[]> {
const entries = await readdir(directory, { recursive: true, withFileTypes: true });
return entries
.filter((entry) => entry.isFile())
.map((entry) =>
path.relative(directory, path.join(entry.parentPath, entry.name)).split(path.sep).join("/"),
)
.sort();
}
async function sourceExists(packageDirectory: string, relative: string): Promise<boolean> {
for (const extension of [".ts", ".tsx"]) {
try {
await stat(path.join(packageDirectory, "src", `${relative}${extension}`));
return true;
} catch {}
}
return false;
}
async function run(command: string, arguments_: readonly string[], cwd: string): Promise<string> {
const { stdout } = await execFileAsync(command, [...arguments_], {
cwd,
maxBuffer: 64 * 1024 * 1024,
});
return stdout;
}
async function inspectTarball(
entry: PublicPackage,
version: string,
tarball: string,
scratch: string,
): Promise<string[]> {
const directory = path.join(repositoryRoot, entry.directory);
const extracted = path.join(scratch, entry.name.replace("/", "+"));
await mkdir(extracted, { recursive: true });
await run("tar", ["-xzf", tarball, "-C", extracted], repositoryRoot);
const root = path.join(extracted, "package");
const files = await filesUnder(root);
const manifest = await readJson(path.join(root, "package.json"));
const source = await readJson(path.join(directory, "package.json"));
const sources = new Map<string, boolean>();
for (const file of files) {
const built = BUILT_FILE.exec(file)?.[1];
if (built !== undefined && !sources.has(built)) {
sources.set(built, await sourceExists(directory, built));
}
}
const problems = [
...packedManifestProblems(manifest, { name: entry.name, version }, source),
...tarballFileProblems(files, manifest, (relative) => sources.get(relative) === true),
];
// Nothing the package says may name the machine or checkout it was packed on.
for (const file of files) {
const text = (await readFile(path.join(root, file))).toString("latin1");
for (const local of [repositoryRoot, homedir(), tmpdir()]) {
if (text.includes(local)) problems.push(`${file} names a local path`);
}
}
return [...new Set(problems)].map((problem) => `${entry.name} ${problem}.`);
}
/**
* Every uncommitted change in the source tree, as `git status --short` lines followed by
* `git diff --stat`, all relative to the repository root; empty for a clean tree. Ignored files
* (build output, `node_modules`) are not source and never count.
*/
async function sourceTreeChanges(): Promise<string[]> {
const status = (
await run("git", ["status", "--short", "--untracked-files=all"], repositoryRoot)
).trimEnd();
if (status === "") return [];
const stat = (await run("git", ["diff", "--stat"], repositoryRoot)).trimEnd();
return [...status.split("\n"), ...(stat === "" ? [] : stat.split("\n"))];
}
export class PackageReleaseError extends Error {
readonly problems: readonly string[];
constructor(problems: readonly string[]) {
super("The package release failed its checks.");
this.name = "PackageReleaseError";
this.problems = problems;
}
}
export interface PackageReleaseOptions {
/** The parent of the release directory, which must not exist yet. */
readonly outDirectory: string;
readonly allowModified?: boolean;
}
/**
* Builds, packs, and inspects the public package train, and writes the release directory only when
* every check passed. Returns its path and manifest; throws `PackageReleaseError` with every problem
* found otherwise.
*/
export async function buildPackageRelease(
options: PackageReleaseOptions,
): Promise<{ readonly directory: string; readonly manifest: PackageTrainManifest }> {
const version = String((await readJson(path.join(repositoryRoot, "package.json"))).version);
const setProblems = releaseSetProblems(await workspacePackages(), version);
if (setProblems.length > 0) throw new PackageReleaseError(setProblems);
const revision = (await run("git", ["rev-parse", "HEAD"], repositoryRoot)).trim();
const dirty = await sourceTreeChanges();
const modified = dirty.length > 0;
if (modified && options.allowModified !== true) {
throw new PackageReleaseError([
"The source tree has uncommitted changes. A release is built from a clean commit.",
...dirty,
]);
}
const release = path.join(path.resolve(options.outDirectory), releaseDirectoryName(version));
try {
await stat(release);
throw new PackageReleaseError([`${release} already exists. A release is written once.`]);
} catch (error) {
if (error instanceof PackageReleaseError) throw error;
}
// Stale output from a deleted source would otherwise ship: the build only ever adds files, and
// Turbo's cache would restore whatever an earlier build left behind.
for (const entry of PUBLIC_PACKAGES) {
await rm(path.join(repositoryRoot, entry.directory, "dist"), { force: true, recursive: true });
}
await run(
"pnpm",
[
"exec",
"turbo",
"run",
"build",
"--force",
"--output-logs=errors-only",
...PUBLIC_PACKAGES.map((entry) => `--filter=${entry.name}`),
],
repositoryRoot,
);
// The source was clean before the build; it must still be, so what is packed is the commit. A build
// that wrote a tracked file, or an untracked file Git does not ignore, fails the release here.
const changedByBuild = (await sourceTreeChanges()).filter((line) => !dirty.includes(line));
if (changedByBuild.length > 0) {
throw new PackageReleaseError([
"The build changed the source tree. A release packs exactly its commit.",
...changedByBuild,
]);
}
const scratch = await mkdtemp(path.join(tmpdir(), "ekklesia-package-release-"));
try {
const staging = path.join(scratch, "release");
await mkdir(staging);
const problems: string[] = [];
const packages: TrainPackage[] = [];
for (const entry of PUBLIC_PACKAGES) {
const report = JSON.parse(
await run(
"pnpm",
[`--config.pnpmfile=${PACK_HOOKS}`, "pack", "--json", "--pack-destination", staging],
path.join(repositoryRoot, entry.directory),
),
) as { filename: string };
const file = path.basename(report.filename);
if (file !== tarballName(entry.name, version)) {
problems.push(`${entry.name} packed as ${file}.`);
continue;
}
problems.push(...(await inspectTarball(entry, version, report.filename, scratch)));
const data = await readFile(report.filename);
packages.push({
file,
name: entry.name,
role: entry.role,
size: data.length,
version,
...artifactDigests(data),
});
}
const manifest: PackageTrainManifest = {
packageTrain: PACKAGE_TRAIN_FORMAT,
packages,
source: { modified, revision },
version,
};
const contents = new Map<string, Uint8Array>();
for (const file of await readdir(staging)) {
contents.set(file, await readFile(path.join(staging, file)));
}
problems.push(...releaseDirectoryProblems(manifest, contents));
if (problems.length > 0) throw new PackageReleaseError(problems);
const sorted = [...manifest.packages].sort((left, right) => (left.file < right.file ? -1 : 1));
await writeFile(
path.join(staging, "SHA256SUMS"),
sorted.map((entry) => `${entry.sha256} ${entry.file}\n`).join(""),
);
await writeFile(path.join(staging, PACKAGE_TRAIN_MANIFEST), serializeManifest(manifest));
await mkdir(path.dirname(release), { recursive: true });
// Into place in one step: the release directory either holds a whole, checked release or does
// not exist. A rename across file systems is not atomic, so copy then, never half a release.
try {
await rename(staging, release);
} catch {
await run("cp", ["-R", staging, release], repositoryRoot);
}
return { directory: release, manifest };
} finally {
await rm(scratch, { force: true, recursive: true });
}
}
async function main(argv: readonly string[]): Promise<number> {
const allowModified = argv.includes("--allow-modified");
const rest = argv.filter((argument) => argument !== "--allow-modified");
if (rest.length !== 2 || rest[0] !== "--out" || !rest[1]) {
process.stderr.write(
"Usage: node scripts/package-release.ts --out <directory> [--allow-modified]\n",
);
return 64;
}
try {
const { directory, manifest } = await buildPackageRelease({
allowModified,
outDirectory: rest[1],
});
for (const entry of manifest.packages) {
process.stdout.write(`${entry.file}: ${entry.size} bytes, sha256 ${entry.sha256}\n`);
}
process.stdout.write(
`Built and inspected the ${manifest.version} package train (${manifest.packages.length} packages${
manifest.source.modified ? ", from a modified source tree: never publish it" : ""
}) in ${directory}\n`,
);
return 0;
} catch (error) {
if (error instanceof PackageReleaseError) {
for (const problem of error.problems) process.stderr.write(`${problem}\n`);
return 1;
}
process.stderr.write("Building the package release failed while building or packing.\n");
return 1;
}
}
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
process.exitCode = await main(process.argv.slice(2));
}
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env bash
# The package release gate (ADR 0053): what must pass before a future step may publish a package
# train. It publishes nothing.
#
# bash scripts/test-package-release.sh [--allow-modified]
#
# 1. Builds the public package train from this checkout (`scripts/package-release.ts`): clean build
# output, pack, inspect, release manifest.
# 2. Builds it again from a separate worktree of the same commit, at another absolute path, and
# requires the two release directories to be byte-identical: every tarball, SHA256SUMS, and the
# manifest.
# 3. Builds Core's verifier, then installs the release into the standalone fixture and proves it
# builds, packages, and verifies (`scripts/test-standalone-module.sh`).
#
# Nothing is built before step 1: the release builder asserts a clean source tree first, builds, and
# asserts the build changed none of it. Every offline install uses the workspace install's own store.
#
# `--allow-modified` is for local development on a modified source tree. The second build is then
# made in this checkout again, because a worktree of the commit would not hold the modifications.
set -euo pipefail
repository_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
allow_modified=()
if [[ "${1:-}" == "--allow-modified" ]]; then
allow_modified=(--allow-modified)
elif (($#)); then
echo "usage: test-package-release.sh [--allow-modified]" >&2
exit 64
fi
# The workspace install's own pnpm store. pnpm chooses a store per file system, so an install in a
# temporary directory would otherwise look in another, empty one; every install here is offline.
store="$(dirname "$(cd "$repository_root" && pnpm store path)")"
export EKKLESIA_PNPM_STORE="$store"
work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-package-release-XXXXXX")"
worktree="$work/second-checkout"
cleanup() {
if [[ -d "$worktree" ]]; then
git -C "$repository_root" worktree remove --force "$worktree" >/dev/null 2>&1 || true
fi
rm -rf "$work"
}
trap cleanup EXIT
echo "==> Building the package train"
node "$repository_root/scripts/package-release.ts" --out "$work/first" "${allow_modified[@]}"
release="$(find "$work/first" -mindepth 1 -maxdepth 1 -type d)"
echo "==> Building it again, to prove the artifacts are reproducible"
if ((${#allow_modified[@]})); then
node "$repository_root/scripts/package-release.ts" --out "$work/second" "${allow_modified[@]}" >/dev/null
else
git -C "$repository_root" worktree add --detach --quiet "$worktree" HEAD
(cd "$worktree" && pnpm install --offline --frozen-lockfile --store-dir "$store" --reporter=silent)
node "$worktree/scripts/package-release.ts" --out "$work/second" >/dev/null
fi
if ! diff -r "$release" "$(find "$work/second" -mindepth 1 -maxdepth 1 -type d)"; then
echo "package release gate failed: two builds of the same commit produced different artifacts" >&2
exit 1
fi
echo " byte-identical: $(ls "$release" | wc -l) files"
# The consumer proof verifies with Core's own verifier. Built only now, after both releases: the
# release builder asserts a clean source tree before it builds anything.
echo "==> Building Core's verifier"
(cd "$repository_root" && pnpm exec turbo run build --filter=@ekklesia/core --output-logs=errors-only >/dev/null)
echo "==> Consuming the release from a standalone module"
bash "$repository_root/scripts/test-standalone-module.sh" --release "$release" "${allow_modified[@]}"
echo "Package release gate passed."
+147 -39
View File
@@ -1,77 +1,175 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# The standalone module-development proof (ADR 0053): a module outside this repository, installed # The standalone consumer proof (ADR 0053): a module outside this repository consumes one package
# from locally packed @ekklesia/* tarballs, with no workspace link and no Ekklesia registry, must # release train, the exact tarballs `scripts/package-release.ts` produced, and nothing else.
# install, typecheck, build its backend and UI, package with the canonical packager, and verify with
# the platform's own verifier.
# #
# bash scripts/test-standalone-module.sh # bash scripts/test-standalone-module.sh --release <release directory> [--allow-modified]
# #
# Third-party packages (React, TypeScript, esbuild, ...) come from the public npm registry as usual. # The release directory is checked against its manifest first; then every @ekklesia/* package, the
# Nothing is published, and nothing outside a temporary directory is written except the packed # module's own and the ones they depend on, installs from those tarballs, with no workspace link, no
# public packages' rebuilt dist/ output. # Ekklesia source checkout, and no registry: third-party packages come from the local pnpm store the
# workspace install already filled (`--offline`). The module then installs again from its lockfile
# alone, type-checks every declaration it reaches (`skipLibCheck: false`), builds its backend and
# UI, packages with the release's own `@ekklesia/module-pack`, and verifies with Core's verifier.
#
# `--allow-modified` accepts a release built from a modified source tree, for local development.
# Core must be built (`core/dist`): the module is verified with Core's own verifier.
set -euo pipefail set -euo pipefail
repository_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" repository_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-standalone-XXXXXX")"
trap 'rm -rf "$work"' EXIT
fail() { fail() {
echo "standalone module proof failed: $*" >&2 echo "standalone consumer proof failed: $*" >&2
exit 1 exit 1
} }
version="$(node -p 'require(process.argv[1]).version' "$repository_root/package.json")" release="" allow_modified=""
while (($#)); do
case "$1" in
--release)
release="${2:-}"
shift 2 || fail "--release needs a directory"
;;
--allow-modified)
allow_modified=1
shift
;;
*) fail "usage: test-standalone-module.sh --release <release directory> [--allow-modified]" ;;
esac
done
[[ -n "$release" && -d "$release" ]] || fail "no release directory given"
echo "==> Packing and inspecting the public package set ($version)" # Third-party packages come from the workspace install's own pnpm store, offline. pnpm chooses a
node "$repository_root/scripts/pack-public-packages.ts" --out "$work/tarballs" # store per file system, so the module, in a temporary directory, is pointed at it explicitly.
store="${EKKLESIA_PNPM_STORE:-$(dirname "$(cd "$repository_root" && pnpm store path)")}"
work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-standalone-XXXXXX")"
trap 'rm -rf "$work"' EXIT
# The module installs from a copy beside it, named by a fixed relative path. pnpm names a file:
# tarball's install directory after that path, and the bundler keeps those names in its comments, so
# an absolute path would make the module package depend on where the release happened to sit. (From
# a registry, the directory is simply `<name>@<version>`.) The copy is what is checked and installed.
cp -R "$release" "$work/release"
release="$work/release"
echo "==> Checking the release against its manifest"
# Prints the train version, then one `name<TAB>file<TAB>integrity` line per package.
node --input-type=module - "$repository_root" "$release" "$allow_modified" >"$work/train.tsv" <<'EOF'
import { readdirSync, readFileSync } from "node:fs";
import path from "node:path";
import { pathToFileURL } from "node:url";
const [repositoryRoot, release, allowModified] = process.argv.slice(2);
const { PACKAGE_TRAIN_MANIFEST, releaseDirectoryProblems } = await import(
pathToFileURL(path.join(repositoryRoot, "scripts", "package-release.ts")).href
);
const manifest = JSON.parse(readFileSync(path.join(release, PACKAGE_TRAIN_MANIFEST), "utf8"));
const files = new Map(readdirSync(release).map((file) => [file, readFileSync(path.join(release, file))]));
const problems = releaseDirectoryProblems(manifest, files);
if (manifest.source?.modified !== false && allowModified !== "1") {
problems.push("the release was built from a modified source tree.");
}
if (problems.length > 0) {
console.error(problems.join("\n"));
process.exit(1);
}
console.log(manifest.version);
for (const entry of manifest.packages) console.log(`${entry.name}\t${entry.file}\t${entry.integrity}`);
EOF
version="$(head -1 "$work/train.tsv")"
echo " package train $version, $(($(wc -l <"$work/train.tsv") - 1)) packages"
module="$work/standalone-proof" module="$work/standalone-proof"
cp -R "$repository_root/tests/fixtures/standalone-module" "$module" cp -R "$repository_root/tests/fixtures/standalone-module" "$module"
cp "$repository_root/LICENSE" "$module/LICENSE" cp "$repository_root/LICENSE" "$module/LICENSE"
# The fixture pins the release's module-pack exactly, as every module repository must.
pinned="$(node -p 'require(process.argv[1]).devDependencies["@ekklesia/module-pack"]' "$module/package.json")"
[[ "$pinned" == "$version" ]] ||
fail "the fixture pins @ekklesia/module-pack $pinned, not the release's $version"
# A module repository's own pnpm settings: esbuild (used by the UI build and the packager) runs its # A module repository's own pnpm settings: esbuild (used by the UI build and the packager) runs its
# install script. And, for this proof only, every @ekklesia/* package, the module's own and the ones # install script. And, for this proof only, every @ekklesia/* package resolves to its release
# they depend on, resolves to its packed tarball, exactly as it would to a published version. # tarball, exactly as it would to the published version.
{ {
echo "allowBuilds:" echo "allowBuilds:"
echo " esbuild: true" echo " esbuild: true"
echo "overrides:" echo "overrides:"
for tarball in "$work"/tarballs/*.tgz; do tail -n +2 "$work/train.tsv" | while IFS=$'\t' read -r name file _; do
name="$(tar -xzOf "$tarball" package/package.json | node -p 'JSON.parse(require("fs").readFileSync(0, "utf8")).name')" echo " \"$name\": \"file:../release/$file\""
echo " \"$name\": \"file:$tarball\""
done done
} >"$module/pnpm-workspace.yaml" } >"$module/pnpm-workspace.yaml"
echo "==> Installing the module from the tarballs" echo "==> Installing the module from the release tarballs, offline"
(cd "$module" && pnpm install --reporter=append-only >"$work/install.log" 2>&1) || (cd "$module" && pnpm install --offline --store-dir "$store" --reporter=append-only >"$work/install.log" 2>&1) ||
{ tail -40 "$work/install.log" >&2; fail "install"; } {
tail -40 "$work/install.log" >&2
fail "install (third-party packages come from the local store: run pnpm install in the workspace first)"
}
echo "==> Checking the lockfile records each release artifact's own integrity"
node --input-type=module - "$module/pnpm-lock.yaml" "$work/train.tsv" <<'EOF'
import { readFileSync } from "node:fs";
const [lockfile, train] = process.argv.slice(2);
const lock = readFileSync(lockfile, "utf8");
const failures = [];
for (const line of readFileSync(train, "utf8").trim().split("\n").slice(1)) {
const [name, file, integrity] = line.split("\t");
// pnpm records a file: tarball under `<name>@file:<path>` with its computed integrity.
const at = lock.indexOf(`\n '${name}@file:`);
const entry = at === -1 ? "" : lock.slice(at, lock.indexOf("\n\n", at));
if (!entry.includes(file)) failures.push(`${name} is not installed from ${file}`);
else if (!entry.includes(`integrity: ${integrity}`)) failures.push(`${name}'s integrity differs from the release manifest`);
}
if (failures.length > 0) {
console.error(failures.join("\n"));
process.exit(1);
}
EOF
echo "==> Reinstalling from the lockfile alone (--frozen-lockfile)"
rm -rf "$module/node_modules"
(cd "$module" && pnpm install --offline --frozen-lockfile --store-dir "$store" --reporter=append-only >"$work/frozen.log" 2>&1) ||
{ tail -40 "$work/frozen.log" >&2; fail "frozen-lockfile install"; }
echo "==> Checking what the install resolved" echo "==> Checking what the install resolved"
(cd "$module" && node --input-type=module - "$module" <<'EOF') (cd "$module" && node --input-type=module - "$module" "$version" <<'EOF')
import { readdirSync, realpathSync } from "node:fs"; import { existsSync, readdirSync, readFileSync, realpathSync } from "node:fs";
import { createRequire } from "node:module"; import { createRequire } from "node:module";
import path from "node:path"; import path from "node:path";
const [moduleDirectory] = process.argv.slice(2); const [moduleDirectory, version] = process.argv.slice(2);
const store = readdirSync(path.join(moduleDirectory, "node_modules", ".pnpm")); const modules = path.join(moduleDirectory, "node_modules");
const store = readdirSync(path.join(modules, ".pnpm"));
const failures = []; const failures = [];
// The Module SDK and its dependencies never pull Drizzle into a module that does not use it. // The Module SDK and its dependencies never pull Drizzle into a module that does not ask for it.
if (store.some((entry) => entry.startsWith("drizzle-orm@"))) failures.push("drizzle-orm is installed"); if (store.some((entry) => entry.startsWith("drizzle-orm@"))) failures.push("drizzle-orm is installed");
// One React, the module's own, shared by @ekklesia/ui. // One React, the module's own, shared by @ekklesia/ui.
const reacts = store.filter((entry) => /^react@/u.test(entry)); const reacts = store.filter((entry) => /^react@/u.test(entry));
if (reacts.length !== 1) failures.push(`${reacts.length} copies of react are installed`); if (reacts.length !== 1) failures.push(`${reacts.length} copies of react are installed`);
const fromModule = createRequire(path.join(moduleDirectory, "package.json")); const fromModule = createRequire(path.join(moduleDirectory, "package.json"));
const ui = realpathSync(path.join(moduleDirectory, "node_modules", "@ekklesia", "ui", "package.json")); const ui = realpathSync(path.join(modules, "@ekklesia", "ui", "package.json"));
const fromUi = createRequire(ui); if (realpathSync(fromModule.resolve("react")) !== realpathSync(createRequire(ui).resolve("react"))) {
if (realpathSync(fromModule.resolve("react")) !== realpathSync(fromUi.resolve("react"))) {
failures.push("@ekklesia/ui resolves a different react"); failures.push("@ekklesia/ui resolves a different react");
} }
// Every Ekklesia package came from a tarball into the module's own store, never from a link. // @ekklesia/contracts is installed only because the packages above depend on it: never a direct
for (const name of readdirSync(path.join(moduleDirectory, "node_modules", "@ekklesia"))) { // dependency, and never reachable by name from the module's own code.
const real = realpathSync(path.join(moduleDirectory, "node_modules", "@ekklesia", name)); const own = JSON.parse(readFileSync(path.join(moduleDirectory, "package.json"), "utf8"));
if (!real.startsWith(path.join(moduleDirectory, "node_modules", ".pnpm") + path.sep)) { for (const field of ["dependencies", "devDependencies", "optionalDependencies", "peerDependencies"]) {
if (own[field]?.["@ekklesia/contracts"] !== undefined) failures.push(`the module lists contracts in ${field}`);
}
if (existsSync(path.join(modules, "@ekklesia", "contracts"))) failures.push("contracts is a direct install");
if (!store.some((entry) => entry.startsWith("@ekklesia+contracts@"))) failures.push("contracts is not installed");
// Every Ekklesia package came from a tarball into the module's own store, never from a link, and
// each one, module-pack included, is exactly the release's version.
for (const name of readdirSync(path.join(modules, "@ekklesia"))) {
const real = realpathSync(path.join(modules, "@ekklesia", name));
if (!real.startsWith(path.join(modules, ".pnpm") + path.sep)) {
failures.push(`@ekklesia/${name} is linked from outside the module`); failures.push(`@ekklesia/${name} is linked from outside the module`);
} }
const installed = JSON.parse(readFileSync(path.join(real, "package.json"), "utf8"));
if (installed.version !== version) failures.push(`@ekklesia/${name} is ${installed.version}`);
} }
if (failures.length > 0) { if (failures.length > 0) {
console.error(failures.join("\n")); console.error(failures.join("\n"));
@@ -95,7 +193,7 @@ echo "==> Typechecking (skipLibCheck: false) and building"
# binary is run directly, because pnpm would re-resolve the module without the proof's overrides. # binary is run directly, because pnpm would re-resolve the module without the proof's overrides.
rm "$module/pnpm-workspace.yaml" rm "$module/pnpm-workspace.yaml"
echo "==> Packaging with the canonical packager" echo "==> Packaging with the release's @ekklesia/module-pack"
(cd "$module" && ./node_modules/.bin/ekklesia-pack-module --module . --target any --out "$work/out" \ (cd "$module" && ./node_modules/.bin/ekklesia-pack-module --module . --target any --out "$work/out" \
--license-file LICENSE --source-revision 86ae907f59458508e3277666cdd214f5bd7c1078) --license-file LICENSE --source-revision 86ae907f59458508e3277666cdd214f5bd7c1078)
archive="$work/out/standalone-proof-0.1.0-any.tar.gz" archive="$work/out/standalone-proof-0.1.0-any.tar.gz"
@@ -105,11 +203,11 @@ echo "==> Verifying with the platform verifier"
--sha256-file "$archive.sha256" --extract-to "$work/verified") --sha256-file "$archive.sha256" --extract-to "$work/verified")
echo "==> Checking the package's own description" echo "==> Checking the package's own description"
node --input-type=module - "$work/verified" "$version" "$repository_root" "$work" <<'EOF' node --input-type=module - "$work/verified" "$version" "$repository_root" "$work" "$release" <<'EOF'
import { readdirSync, readFileSync } from "node:fs"; import { readdirSync, readFileSync } from "node:fs";
import path from "node:path"; import path from "node:path";
const [verified, version, repositoryRoot, work] = process.argv.slice(2); const [verified, version, repositoryRoot, work, release] = process.argv.slice(2);
const read = (file) => readFileSync(path.join(verified, file), "utf8"); const read = (file) => readFileSync(path.join(verified, file), "utf8");
const metadata = JSON.parse(read("ekklesia.package.json")); const metadata = JSON.parse(read("ekklesia.package.json"));
const failures = []; const failures = [];
@@ -117,6 +215,10 @@ const incorporates = metadata.incorporates ?? {};
for (const name of ["@ekklesia/contracts", "@ekklesia/module-database", "@ekklesia/module-sdk"]) { for (const name of ["@ekklesia/contracts", "@ekklesia/module-database", "@ekklesia/module-sdk"]) {
if (incorporates[name] !== version) failures.push(`incorporates ${name}@${incorporates[name]}`); if (incorporates[name] !== version) failures.push(`incorporates ${name}@${incorporates[name]}`);
} }
const packager = metadata.provenance?.packager ?? {};
if (packager.name !== "@ekklesia/module-pack" || packager.version !== version) {
failures.push(`packed by ${packager.name}@${packager.version}, not the release's module-pack`);
}
if (metadata.runtime?.node !== ">=24.11.0 <25") failures.push("runtime.node is not engines.node"); if (metadata.runtime?.node !== ">=24.11.0 <25") failures.push("runtime.node is not engines.node");
if (metadata.transitional !== undefined) failures.push("records transitional database facts"); if (metadata.transitional !== undefined) failures.push("records transitional database facts");
const notices = read("THIRD_PARTY_NOTICES"); const notices = read("THIRD_PARTY_NOTICES");
@@ -127,8 +229,14 @@ if (reacts.length !== 1) failures.push(`credits ${reacts.length} copies of react
for (const entry of readdirSync(verified, { recursive: true, withFileTypes: true })) { for (const entry of readdirSync(verified, { recursive: true, withFileTypes: true })) {
if (!entry.isFile()) continue; if (!entry.isFile()) continue;
const text = readFileSync(path.join(entry.parentPath, entry.name), "latin1"); const text = readFileSync(path.join(entry.parentPath, entry.name), "latin1");
if (text.includes(repositoryRoot) || text.includes(work)) failures.push(`${entry.name} names a local path`); for (const local of [repositoryRoot, work, release]) {
if (text.includes(local)) failures.push(`${entry.name} names a local path`);
}
if (/drizzle-orm/u.test(text) && entry.name.endsWith(".js")) failures.push(`${entry.name} bundles Drizzle`); if (/drizzle-orm/u.test(text) && entry.name.endsWith(".js")) failures.push(`${entry.name} bundles Drizzle`);
// A file: install directory's name keeps the path pnpm installed it from: only the fixed one.
for (const match of text.matchAll(/@file\+([^/"'\s]*)/gu)) {
if (!match[1]?.startsWith("..+release+")) failures.push(`${entry.name} names an install path`);
}
} }
if (failures.length > 0) { if (failures.length > 0) {
console.error(failures.join("\n")); console.error(failures.join("\n"));
@@ -137,4 +245,4 @@ if (failures.length > 0) {
console.log(`incorporates ${JSON.stringify(incorporates)}`); console.log(`incorporates ${JSON.stringify(incorporates)}`);
EOF EOF
echo "Standalone module proof passed." echo "Standalone consumer proof passed for package train $version."
+174 -8
View File
@@ -1,20 +1,29 @@
import { spawnSync } from "node:child_process";
import { readdir, readFile } from "node:fs/promises"; import { readdir, readFile } from "node:fs/promises";
import path from "node:path"; import path from "node:path";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { parse } from "yaml"; import { parse } from "yaml";
import { import {
artifactDigests,
PACKAGE_TRAIN_MANIFEST,
type PackageTrainManifest,
PUBLIC_PACKAGE_DISTRIBUTION, PUBLIC_PACKAGE_DISTRIBUTION,
PUBLIC_PACKAGES, PUBLIC_PACKAGES,
packedManifestProblems, packedManifestProblems,
releaseDirectoryProblems,
releaseSetProblems,
serializeManifest,
tarballFileProblems, tarballFileProblems,
} from "../scripts/pack-public-packages.js"; tarballName,
workspacePackages as releaseWorkspacePackages,
} from "../scripts/package-release.js";
/** /**
* The public module-development package boundary (ADR 0053): which packages may be published, what * The public module-development package boundary and its release train (ADR 0053): which packages
* their manifests may say, and what the author-facing SDK exposes. The tarballs themselves are * may be published, what their manifests may say, what the author-facing SDK exposes, and what a
* inspected by `pnpm packages:pack` and installed into a standalone module by * release directory must hold. The tarballs themselves are built and inspected by
* `scripts/test-standalone-module.sh`. * `pnpm packages:release` (`scripts/package-release.ts`), and proven by `pnpm test:package-release`.
*/ */
const repositoryRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), ".."); const repositoryRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..");
@@ -78,6 +87,41 @@ describe("the public package set", () => {
expect(packages.get("@ekklesia/module-template")?.manifest.private).toBe(true); expect(packages.get("@ekklesia/module-template")?.manifest.private).toBe(true);
}); });
it("is what the release builder itself accepts, read without a YAML parser", async () => {
const platformVersion = (await readJson("package.json")).version;
const read = await releaseWorkspacePackages(repositoryRoot);
expect(read.map((entry) => entry.name).sort()).toEqual(
[...(await workspacePackages()).keys()].sort(),
);
expect(releaseSetProblems(read, platformVersion)).toEqual([]);
});
it("refuses a missing, mis-versioned, or private public package, and anything else publishable", async () => {
const read = await releaseWorkspacePackages(repositoryRoot);
const without = read.filter((entry) => entry.name !== "@ekklesia/ui");
expect(releaseSetProblems(without, "0.1.2").join(" ")).toContain("@ekklesia/ui is missing");
const changed = read.map((entry) =>
entry.name === "@ekklesia/module-sdk"
? { ...entry, manifest: { ...entry.manifest, private: true, version: "0.2.0" } }
: entry.name === "@ekklesia/core"
? { ...entry, manifest: { ...entry.manifest, private: undefined } }
: entry.name === "@ekklesia/module-ui-host"
? {
...entry,
manifest: {
...entry.manifest,
ekklesia: { distribution: PUBLIC_PACKAGE_DISTRIBUTION },
},
}
: entry,
);
const problems = releaseSetProblems(changed, "0.1.2").join(" ");
expect(problems).toContain("@ekklesia/module-sdk is version 0.2.0");
expect(problems).toContain("@ekklesia/module-sdk is private");
expect(problems).toContain("@ekklesia/core is publishable but not in the release set");
expect(problems).toContain("@ekklesia/module-ui-host carries the distribution marker");
});
it("uses one coordinated version, and depends only on other public packages through the workspace", async () => { it("uses one coordinated version, and depends only on other public packages through the workspace", async () => {
const platformVersion = (await readJson("package.json")).version; const platformVersion = (await readJson("package.json")).version;
const publicNames = new Set(PUBLIC_PACKAGES.map((entry) => entry.name)); const publicNames = new Set(PUBLIC_PACKAGES.map((entry) => entry.name));
@@ -208,10 +252,11 @@ describe("module authors' dependencies", () => {
} }
}); });
it("pin module-pack exactly and declare engines.node in the standalone fixture", async () => { it("pin module-pack exactly, at this release, and declare engines.node in the standalone fixture", async () => {
const fixture = await readJson("tests/fixtures/standalone-module/package.json"); const fixture = await readJson("tests/fixtures/standalone-module/package.json");
const devDependencies = fixture.devDependencies as Record<string, string>; const devDependencies = fixture.devDependencies as Record<string, string>;
expect(devDependencies["@ekklesia/module-pack"]).toMatch(/^\d+\.\d+\.\d+$/u); // Part of setting a release version: the fixture consumes exactly this release's module-pack.
expect(devDependencies["@ekklesia/module-pack"]).toBe((await readJson("package.json")).version);
expect((fixture.engines as Record<string, string>).node).toBeTruthy(); expect((fixture.engines as Record<string, string>).node).toBeTruthy();
}); });
}); });
@@ -231,7 +276,6 @@ describe("@ekklesia/typescript-config", () => {
describe("the tarball inspection", () => { describe("the tarball inspection", () => {
const good = { const good = {
dependencies: { "@ekklesia/contracts": "0.1.2", zod: "4.4.3" }, dependencies: { "@ekklesia/contracts": "0.1.2", zod: "4.4.3" },
devDependencies: { "@ekklesia/core": "0.1.2" },
ekklesia: { distribution: PUBLIC_PACKAGE_DISTRIBUTION, layer: "module-sdk" }, ekklesia: { distribution: PUBLIC_PACKAGE_DISTRIBUTION, layer: "module-sdk" },
files: ["dist"], files: ["dist"],
license: "AGPL-3.0-or-later", license: "AGPL-3.0-or-later",
@@ -255,6 +299,10 @@ describe("the tarball inspection", () => {
{ ...good, dependencies: { "@ekklesia/contracts": "0.1.1" } }, { ...good, dependencies: { "@ekklesia/contracts": "0.1.1" } },
{ ...good, peerDependencies: { "@ekklesia/core": "0.1.2" } }, { ...good, peerDependencies: { "@ekklesia/core": "0.1.2" } },
{ ...good, version: "0.1.3" }, { ...good, version: "0.1.3" },
// Development dependencies never ship: they name private packages outside the train.
{ ...good, devDependencies: { "@ekklesia/core": "0.1.2" } },
// A registry is the publication step's configuration, never part of an artifact.
{ ...good, publishConfig: { registry: "https://registry.example/" } },
]; ];
for (const manifest of cases) { for (const manifest of cases) {
expect(packedManifestProblems(manifest, expected), JSON.stringify(manifest)).not.toEqual([]); expect(packedManifestProblems(manifest, expected), JSON.stringify(manifest)).not.toEqual([]);
@@ -280,4 +328,122 @@ describe("the tarball inspection", () => {
expect(tarballFileProblems(files, good, hasSource), files.join(" ")).not.toEqual([]); expect(tarballFileProblems(files, good, hasSource), files.join(" ")).not.toEqual([]);
} }
}); });
it("refuses a dependency map whose order differs from its source", () => {
const source = { dependencies: { "@ekklesia/contracts": "workspace:*", zod: "4.4.3" } };
expect(packedManifestProblems(good, expected, source)).toEqual([]);
const reordered = { ...good, dependencies: { zod: "4.4.3", "@ekklesia/contracts": "0.1.2" } };
expect(packedManifestProblems(reordered, expected, source)).toEqual([
"orders dependencies differently from its source",
]);
});
});
describe("the pack hook", () => {
it("drops devDependencies and restores the source's key order, changing nothing else", async () => {
const { hooks } = await import("../scripts/lib/pack-hooks.cjs");
const directory = path.join(repositoryRoot, "packages", "module-pack");
const source = await readJson("packages/module-pack/package.json");
const dependencies = Object.fromEntries(
Object.keys(source.dependencies ?? {})
.reverse()
.map((name) => [name, name.startsWith("@ekklesia/") ? "0.1.2" : "1.0.0"]),
);
const packed = hooks.beforePacking(
{ ...source, dependencies, devDependencies: { "@ekklesia/core": "0.1.2" } },
directory,
);
expect(packed.devDependencies).toBeUndefined();
expect(Object.keys(packed.dependencies as object)).toEqual(
Object.keys(source.dependencies ?? {}),
);
expect(packed.dependencies).toEqual(dependencies);
expect(packed.name).toBe(source.name);
});
});
describe("a clean source tree", () => {
it("ignores pnpm's package store, which pnpm creates in a checkout that is its own mount", () => {
// In CI the checkout is a workspace volume, so `pnpm install` puts `.pnpm-store/` at its root:
// a package cache, never source, which must not make the release builder's clean tree dirty.
const ignored = spawnSync("git", ["check-ignore", "--quiet", ".pnpm-store/v11/index.db"], {
cwd: repositoryRoot,
});
expect(ignored.status).toBe(0);
});
});
describe("the release directory", () => {
const version = "0.1.2";
function release(mutate?: (files: Map<string, Uint8Array>) => void) {
const files = new Map<string, Uint8Array>();
for (const entry of PUBLIC_PACKAGES) {
files.set(tarballName(entry.name, version), Buffer.from(`tarball of ${entry.name}`));
}
const manifest: PackageTrainManifest = {
packageTrain: 1,
packages: PUBLIC_PACKAGES.map((entry) => {
const file = tarballName(entry.name, version);
const data = files.get(file) ?? new Uint8Array();
return {
file,
name: entry.name,
role: entry.role,
size: data.length,
version,
...artifactDigests(data),
};
}),
source: { modified: false, revision: "86ae907f59458508e3277666cdd214f5bd7c1078" },
version,
};
mutate?.(files);
files.set(PACKAGE_TRAIN_MANIFEST, Buffer.from(serializeManifest(manifest)));
return { files, manifest };
}
it("names each tarball as pnpm pack does", () => {
expect(tarballName("@ekklesia/module-sdk", "0.1.2")).toBe("ekklesia-module-sdk-0.1.2.tgz");
});
it("records npm-compatible integrity and SHA-256 for each artifact", () => {
const digests = artifactDigests(Buffer.from("abc"));
expect(digests.sha256).toBe("ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad");
expect(digests.integrity).toMatch(/^sha512-[A-Za-z0-9+/]{86}==$/u);
});
it("accepts exactly the expected train, matching its manifest", () => {
const { files, manifest } = release();
expect(releaseDirectoryProblems(manifest, files)).toEqual([]);
});
it("refuses a missing, extra, or altered artifact", () => {
const missing = release((files) => files.delete(tarballName("@ekklesia/ui", version)));
expect(releaseDirectoryProblems(missing.manifest, missing.files).join(" ")).toContain(
"ekklesia-ui-0.1.2.tgz is missing",
);
const extra = release((files) =>
files.set("ekklesia-module-ui-host-0.1.2.tgz", Buffer.from("x")),
);
expect(releaseDirectoryProblems(extra.manifest, extra.files).join(" ")).toContain(
"ekklesia-module-ui-host-0.1.2.tgz is not part of the release",
);
const altered = release((files) =>
files.set(tarballName("@ekklesia/module-pack", version), Buffer.from("changed")),
);
expect(releaseDirectoryProblems(altered.manifest, altered.files).join(" ")).toContain(
"does not match its size, SHA-256, or integrity",
);
});
it("serializes the manifest the same way every time, packages sorted by name", () => {
const { manifest } = release();
const reversed = { ...manifest, packages: [...manifest.packages].reverse() };
expect(serializeManifest(reversed)).toBe(serializeManifest(manifest));
const parsed = JSON.parse(serializeManifest(manifest)) as PackageTrainManifest;
expect(parsed.packages.map((entry) => entry.name)).toEqual(
[...PUBLIC_PACKAGES.map((entry) => entry.name)].sort(),
);
expect(Object.keys(parsed)).toEqual(["packageTrain", "version", "source", "packages"]);
});
}); });