fix: keep pnpm's package store out of the release builder's clean-source check
CI / verify (pull_request) Successful in 1m4s
CI / infrastructure-smoke (pull_request) Successful in 37s
CI / migration-smoke (pull_request) Successful in 1m54s
CI / package-release (pull_request) Successful in 32s
CI / image-build (pull_request) Successful in 30s
CI / deploy-smoke (same-origin) (pull_request) Successful in 50s
CI / deploy-smoke (split) (pull_request) Successful in 1m32s

The package-release job failed its clean-tree assertion in CI. Reproduced
as root in a container with a bind-mounted checkout, as the Gitea runner
does: after checkout the tree is clean; `pnpm install --frozen-lockfile`
then creates `.pnpm-store/` (17,272 untracked files) at the checkout's
root, because pnpm keeps its store on the node_modules file system and
the checkout is its own mount; `pnpm build` changes nothing. No tracked
file is ever modified.

- `.pnpm-store/` is ignored, like node_modules: a package cache, never
  source. The clean-commit requirement is unchanged for every tracked
  file and every untracked file Git does not ignore.
- The clean-tree failure now lists `git status --short` and
  `git diff --stat`.
- Nothing is built before the clean-source assertion: the gate no longer
  runs a global `pnpm build` first. The builder asserts a clean tree,
  builds, then fails if its build changed the tree, before packing. Core's
  verifier is built after both releases, for the consumer proof only.
- Every secondary offline install (the second worktree, the consumer
  module) uses the workspace install's own store explicitly, since pnpm
  picks a store per file system and those directories are on another.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-10-08 08:43:19 -05:00
co-authored by Claude Opus 5.5
parent 9debe4e891
commit 4f1b1d68c7
7 changed files with 80 additions and 17 deletions
+4
View File
@@ -1,4 +1,8 @@
node_modules/ node_modules/
# pnpm's content-addressed package store. pnpm keeps it beside node_modules on the same file system,
# so when the checkout is its own mount (a CI runner's workspace volume, a bind-mounted container),
# `pnpm install` creates it here, at the checkout's root. A package cache, never source.
.pnpm-store/
.turbo/ .turbo/
coverage/ coverage/
dist/ dist/
+14 -7
View File
@@ -63,12 +63,18 @@ pnpm packages:release # writes .package-release/ekklesia-packages-<vers
`scripts/package-release.ts`: `scripts/package-release.ts`:
1. **Refuses a modified source tree.** A release is built from a commit. `--allow-modified` exists 1. **Refuses a modified source tree, before it builds anything.** A release is built from a
for local development only; its manifest then says `"modified": true`, and such a release must commit. A tree is clean when `git status` shows no change to a tracked file and no untracked file
never be published. that Git does not ignore. Ignored files are build output and package-manager state:
`node_modules`, `dist`, `.turbo`, and pnpm's `.pnpm-store`, which pnpm creates at the checkout's
root when the checkout is its own mount (a CI runner's workspace volume). The failure lists every
change. `--allow-modified` exists for local development only; its manifest then says
`"modified": true`, and such a release must never be published.
2. **Checks the release set and versions** in the workspace. 2. **Checks the release set and versions** in the workspace.
3. **Rebuilds from clean output.** It removes each public package's `dist/` and rebuilds without the 3. **Rebuilds from clean output.** It removes each public package's `dist/` and rebuilds without the
build cache, so output from a deleted source file can never ship. build cache, so output from a deleted source file can never ship. It then checks the source tree
again: a build that changed a tracked file, or wrote an untracked one Git does not ignore, fails
the release, because what is packed must be exactly the commit.
4. **Packs each package with `pnpm pack`.** Pack-only pnpm hooks (`scripts/lib/pack-hooks.cjs`) 4. **Packs each package with `pnpm pack`.** Pack-only pnpm hooks (`scripts/lib/pack-hooks.cjs`)
apply, never the workspace's install settings ([Reproducibility](#reproducibility)). apply, never the workspace's install settings ([Reproducibility](#reproducibility)).
5. **Inspects every tarball.** It checks: 5. **Inspects every tarball.** It checks:
@@ -127,11 +133,11 @@ depend only on the commit.
pnpm test:package-release # scripts/test-package-release.sh pnpm test:package-release # scripts/test-package-release.sh
``` ```
1. **Builds the train** from this checkout. 1. **Builds the train** from this checkout. Nothing is built before it.
2. **Builds it again** from a second worktree of the same commit at another absolute path, and 2. **Builds it again** from a second worktree of the same commit at another absolute path, and
requires both release directories to be byte-identical: every tarball, `SHA256SUMS`, and the requires both release directories to be byte-identical: every tarball, `SHA256SUMS`, and the
manifest. manifest.
3. **Runs the standalone consumer proof** (`scripts/test-standalone-module.sh --release <dir>`) on 3. **Builds Core's verifier**, then **runs the standalone consumer proof** (`scripts/test-standalone-module.sh --release <dir>`) on
that release: that release:
- **The release is checked first.** Every artifact must match the manifest's size, SHA-256, and - **The release is checked first.** Every artifact must match the manifest's size, SHA-256, and
integrity, and the release must not be built from a modified tree. integrity, and the release must not be built from a modified tree.
@@ -139,7 +145,8 @@ pnpm test:package-release # scripts/test-package-release.sh
`tests/fixtures/standalone-module`, copied outside the repository. Every `@ekklesia/*` `tests/fixtures/standalone-module`, copied outside the repository. Every `@ekklesia/*`
package, direct or transitive, resolves to its release tarball. There is no workspace link and package, direct or transitive, resolves to its release tarball. There is no workspace link and
no Ekklesia source checkout. The install runs offline: third-party packages come from the no Ekklesia source checkout. The install runs offline: third-party packages come from the
local pnpm store, so no registry is involved at all. workspace install's own pnpm store, named explicitly because pnpm otherwise picks a store per
file system, so no registry is involved at all.
- **The lockfile matches the release.** It must record each artifact's own integrity. The module - **The lockfile matches the release.** It must record each artifact's own integrity. The module
then reinstalls from the lockfile alone with `--frozen-lockfile`. then reinstalls from the lockfile alone with `--frozen-lockfile`.
- **The install has the right shape.** There is no `drizzle-orm`, and exactly one `react`, which - **The install has the right shape.** There is no `drizzle-orm`, and exactly one `react`, which
+1 -1
View File
@@ -42,7 +42,7 @@
"test": "turbo run test test:root", "test": "turbo run test test:root",
"test:migrations:integration": "pnpm build && bash scripts/test-migrations-integration.sh", "test:migrations:integration": "pnpm build && bash scripts/test-migrations-integration.sh",
"test:root": "vitest run --config vitest.config.ts", "test:root": "vitest run --config vitest.config.ts",
"test:package-release": "pnpm build && bash scripts/test-package-release.sh", "test:package-release": "bash scripts/test-package-release.sh",
"test:watch": "vitest", "test:watch": "vitest",
"typecheck": "turbo run typecheck typecheck:root", "typecheck": "turbo run typecheck typecheck:root",
"typecheck:root": "tsc --noEmit --project tsconfig.json", "typecheck:root": "tsc --noEmit --project tsconfig.json",
+26 -4
View File
@@ -433,6 +433,20 @@ async function inspectTarball(
return [...new Set(problems)].map((problem) => `${entry.name} ${problem}.`); return [...new Set(problems)].map((problem) => `${entry.name} ${problem}.`);
} }
/**
* Every uncommitted change in the source tree, as `git status --short` lines followed by
* `git diff --stat`, all relative to the repository root; empty for a clean tree. Ignored files
* (build output, `node_modules`) are not source and never count.
*/
async function sourceTreeChanges(): Promise<string[]> {
const status = (
await run("git", ["status", "--short", "--untracked-files=all"], repositoryRoot)
).trimEnd();
if (status === "") return [];
const stat = (await run("git", ["diff", "--stat"], repositoryRoot)).trimEnd();
return [...status.split("\n"), ...(stat === "" ? [] : stat.split("\n"))];
}
export class PackageReleaseError extends Error { export class PackageReleaseError extends Error {
readonly problems: readonly string[]; readonly problems: readonly string[];
@@ -462,13 +476,12 @@ export async function buildPackageRelease(
if (setProblems.length > 0) throw new PackageReleaseError(setProblems); if (setProblems.length > 0) throw new PackageReleaseError(setProblems);
const revision = (await run("git", ["rev-parse", "HEAD"], repositoryRoot)).trim(); const revision = (await run("git", ["rev-parse", "HEAD"], repositoryRoot)).trim();
const modified = const dirty = await sourceTreeChanges();
( const modified = dirty.length > 0;
await run("git", ["status", "--porcelain", "--untracked-files=all"], repositoryRoot)
).trim() !== "";
if (modified && options.allowModified !== true) { if (modified && options.allowModified !== true) {
throw new PackageReleaseError([ throw new PackageReleaseError([
"The source tree has uncommitted changes. A release is built from a clean commit.", "The source tree has uncommitted changes. A release is built from a clean commit.",
...dirty,
]); ]);
} }
@@ -498,6 +511,15 @@ export async function buildPackageRelease(
], ],
repositoryRoot, repositoryRoot,
); );
// The source was clean before the build; it must still be, so what is packed is the commit. A build
// that wrote a tracked file, or an untracked file Git does not ignore, fails the release here.
const changedByBuild = (await sourceTreeChanges()).filter((line) => !dirty.includes(line));
if (changedByBuild.length > 0) {
throw new PackageReleaseError([
"The build changed the source tree. A release packs exactly its commit.",
...changedByBuild,
]);
}
const scratch = await mkdtemp(path.join(tmpdir(), "ekklesia-package-release-")); const scratch = await mkdtemp(path.join(tmpdir(), "ekklesia-package-release-"));
try { try {
+16 -3
View File
@@ -9,8 +9,11 @@
# 2. Builds it again from a separate worktree of the same commit, at another absolute path, and # 2. Builds it again from a separate worktree of the same commit, at another absolute path, and
# requires the two release directories to be byte-identical: every tarball, SHA256SUMS, and the # requires the two release directories to be byte-identical: every tarball, SHA256SUMS, and the
# manifest. # manifest.
# 3. Installs the release into the standalone fixture and proves it builds, packages, and verifies # 3. Builds Core's verifier, then installs the release into the standalone fixture and proves it
# (`scripts/test-standalone-module.sh`). # builds, packages, and verifies (`scripts/test-standalone-module.sh`).
#
# Nothing is built before step 1: the release builder asserts a clean source tree first, builds, and
# asserts the build changed none of it. Every offline install uses the workspace install's own store.
# #
# `--allow-modified` is for local development on a modified source tree. The second build is then # `--allow-modified` is for local development on a modified source tree. The second build is then
# made in this checkout again, because a worktree of the commit would not hold the modifications. # made in this checkout again, because a worktree of the commit would not hold the modifications.
@@ -25,6 +28,11 @@ elif (($#)); then
exit 64 exit 64
fi fi
# The workspace install's own pnpm store. pnpm chooses a store per file system, so an install in a
# temporary directory would otherwise look in another, empty one; every install here is offline.
store="$(dirname "$(cd "$repository_root" && pnpm store path)")"
export EKKLESIA_PNPM_STORE="$store"
work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-package-release-XXXXXX")" work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-package-release-XXXXXX")"
worktree="$work/second-checkout" worktree="$work/second-checkout"
cleanup() { cleanup() {
@@ -44,7 +52,7 @@ if ((${#allow_modified[@]})); then
node "$repository_root/scripts/package-release.ts" --out "$work/second" "${allow_modified[@]}" >/dev/null node "$repository_root/scripts/package-release.ts" --out "$work/second" "${allow_modified[@]}" >/dev/null
else else
git -C "$repository_root" worktree add --detach --quiet "$worktree" HEAD git -C "$repository_root" worktree add --detach --quiet "$worktree" HEAD
(cd "$worktree" && pnpm install --offline --frozen-lockfile --reporter=silent) (cd "$worktree" && pnpm install --offline --frozen-lockfile --store-dir "$store" --reporter=silent)
node "$worktree/scripts/package-release.ts" --out "$work/second" >/dev/null node "$worktree/scripts/package-release.ts" --out "$work/second" >/dev/null
fi fi
if ! diff -r "$release" "$(find "$work/second" -mindepth 1 -maxdepth 1 -type d)"; then if ! diff -r "$release" "$(find "$work/second" -mindepth 1 -maxdepth 1 -type d)"; then
@@ -53,6 +61,11 @@ if ! diff -r "$release" "$(find "$work/second" -mindepth 1 -maxdepth 1 -type d)"
fi fi
echo " byte-identical: $(ls "$release" | wc -l) files" echo " byte-identical: $(ls "$release" | wc -l) files"
# The consumer proof verifies with Core's own verifier. Built only now, after both releases: the
# release builder asserts a clean source tree before it builds anything.
echo "==> Building Core's verifier"
(cd "$repository_root" && pnpm exec turbo run build --filter=@ekklesia/core --output-logs=errors-only >/dev/null)
echo "==> Consuming the release from a standalone module" echo "==> Consuming the release from a standalone module"
bash "$repository_root/scripts/test-standalone-module.sh" --release "$release" "${allow_modified[@]}" bash "$repository_root/scripts/test-standalone-module.sh" --release "$release" "${allow_modified[@]}"
+7 -2
View File
@@ -12,6 +12,7 @@
# UI, packages with the release's own `@ekklesia/module-pack`, and verifies with Core's verifier. # UI, packages with the release's own `@ekklesia/module-pack`, and verifies with Core's verifier.
# #
# `--allow-modified` accepts a release built from a modified source tree, for local development. # `--allow-modified` accepts a release built from a modified source tree, for local development.
# Core must be built (`core/dist`): the module is verified with Core's own verifier.
set -euo pipefail set -euo pipefail
repository_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" repository_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@@ -37,6 +38,10 @@ while (($#)); do
done done
[[ -n "$release" && -d "$release" ]] || fail "no release directory given" [[ -n "$release" && -d "$release" ]] || fail "no release directory given"
# Third-party packages come from the workspace install's own pnpm store, offline. pnpm chooses a
# store per file system, so the module, in a temporary directory, is pointed at it explicitly.
store="${EKKLESIA_PNPM_STORE:-$(dirname "$(cd "$repository_root" && pnpm store path)")}"
work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-standalone-XXXXXX")" work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-standalone-XXXXXX")"
trap 'rm -rf "$work"' EXIT trap 'rm -rf "$work"' EXIT
@@ -96,7 +101,7 @@ pinned="$(node -p 'require(process.argv[1]).devDependencies["@ekklesia/module-pa
} >"$module/pnpm-workspace.yaml" } >"$module/pnpm-workspace.yaml"
echo "==> Installing the module from the release tarballs, offline" echo "==> Installing the module from the release tarballs, offline"
(cd "$module" && pnpm install --offline --reporter=append-only >"$work/install.log" 2>&1) || (cd "$module" && pnpm install --offline --store-dir "$store" --reporter=append-only >"$work/install.log" 2>&1) ||
{ {
tail -40 "$work/install.log" >&2 tail -40 "$work/install.log" >&2
fail "install (third-party packages come from the local store: run pnpm install in the workspace first)" fail "install (third-party packages come from the local store: run pnpm install in the workspace first)"
@@ -125,7 +130,7 @@ EOF
echo "==> Reinstalling from the lockfile alone (--frozen-lockfile)" echo "==> Reinstalling from the lockfile alone (--frozen-lockfile)"
rm -rf "$module/node_modules" rm -rf "$module/node_modules"
(cd "$module" && pnpm install --offline --frozen-lockfile --reporter=append-only >"$work/frozen.log" 2>&1) || (cd "$module" && pnpm install --offline --frozen-lockfile --store-dir "$store" --reporter=append-only >"$work/frozen.log" 2>&1) ||
{ tail -40 "$work/frozen.log" >&2; fail "frozen-lockfile install"; } { tail -40 "$work/frozen.log" >&2; fail "frozen-lockfile install"; }
echo "==> Checking what the install resolved" echo "==> Checking what the install resolved"
+12
View File
@@ -1,3 +1,4 @@
import { spawnSync } from "node:child_process";
import { readdir, readFile } from "node:fs/promises"; import { readdir, readFile } from "node:fs/promises";
import path from "node:path"; import path from "node:path";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
@@ -361,6 +362,17 @@ describe("the pack hook", () => {
}); });
}); });
describe("a clean source tree", () => {
it("ignores pnpm's package store, which pnpm creates in a checkout that is its own mount", () => {
// In CI the checkout is a workspace volume, so `pnpm install` puts `.pnpm-store/` at its root:
// a package cache, never source, which must not make the release builder's clean tree dirty.
const ignored = spawnSync("git", ["check-ignore", "--quiet", ".pnpm-store/v11/index.db"], {
cwd: repositoryRoot,
});
expect(ignored.status).toBe(0);
});
});
describe("the release directory", () => { describe("the release directory", () => {
const version = "0.1.2"; const version = "0.1.2";
function release(mutate?: (files: Map<string, Uint8Array>) => void) { function release(mutate?: (files: Map<string, Uint8Array>) => void) {