fix: keep pnpm's package store out of the release builder's clean-source check
CI / verify (pull_request) Successful in 1m4s
CI / infrastructure-smoke (pull_request) Successful in 37s
CI / migration-smoke (pull_request) Successful in 1m54s
CI / package-release (pull_request) Successful in 32s
CI / image-build (pull_request) Successful in 30s
CI / deploy-smoke (same-origin) (pull_request) Successful in 50s
CI / deploy-smoke (split) (pull_request) Successful in 1m32s

The package-release job failed its clean-tree assertion in CI. Reproduced
as root in a container with a bind-mounted checkout, as the Gitea runner
does: after checkout the tree is clean; `pnpm install --frozen-lockfile`
then creates `.pnpm-store/` (17,272 untracked files) at the checkout's
root, because pnpm keeps its store on the node_modules file system and
the checkout is its own mount; `pnpm build` changes nothing. No tracked
file is ever modified.

- `.pnpm-store/` is ignored, like node_modules: a package cache, never
  source. The clean-commit requirement is unchanged for every tracked
  file and every untracked file Git does not ignore.
- The clean-tree failure now lists `git status --short` and
  `git diff --stat`.
- Nothing is built before the clean-source assertion: the gate no longer
  runs a global `pnpm build` first. The builder asserts a clean tree,
  builds, then fails if its build changed the tree, before packing. Core's
  verifier is built after both releases, for the consumer proof only.
- Every secondary offline install (the second worktree, the consumer
  module) uses the workspace install's own store explicitly, since pnpm
  picks a store per file system and those directories are on another.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
ekklesia
2026-10-08 08:43:19 -05:00
co-authored by Claude Opus 5.5
parent 9debe4e891
commit 4f1b1d68c7
7 changed files with 80 additions and 17 deletions
+4
View File
@@ -1,4 +1,8 @@
node_modules/
# pnpm's content-addressed package store. pnpm keeps it beside node_modules on the same file system,
# so when the checkout is its own mount (a CI runner's workspace volume, a bind-mounted container),
# `pnpm install` creates it here, at the checkout's root. A package cache, never source.
.pnpm-store/
.turbo/
coverage/
dist/
+14 -7
View File
@@ -63,12 +63,18 @@ pnpm packages:release # writes .package-release/ekklesia-packages-<vers
`scripts/package-release.ts`:
1. **Refuses a modified source tree.** A release is built from a commit. `--allow-modified` exists
for local development only; its manifest then says `"modified": true`, and such a release must
never be published.
1. **Refuses a modified source tree, before it builds anything.** A release is built from a
commit. A tree is clean when `git status` shows no change to a tracked file and no untracked file
that Git does not ignore. Ignored files are build output and package-manager state:
`node_modules`, `dist`, `.turbo`, and pnpm's `.pnpm-store`, which pnpm creates at the checkout's
root when the checkout is its own mount (a CI runner's workspace volume). The failure lists every
change. `--allow-modified` exists for local development only; its manifest then says
`"modified": true`, and such a release must never be published.
2. **Checks the release set and versions** in the workspace.
3. **Rebuilds from clean output.** It removes each public package's `dist/` and rebuilds without the
build cache, so output from a deleted source file can never ship.
build cache, so output from a deleted source file can never ship. It then checks the source tree
again: a build that changed a tracked file, or wrote an untracked one Git does not ignore, fails
the release, because what is packed must be exactly the commit.
4. **Packs each package with `pnpm pack`.** Pack-only pnpm hooks (`scripts/lib/pack-hooks.cjs`)
apply, never the workspace's install settings ([Reproducibility](#reproducibility)).
5. **Inspects every tarball.** It checks:
@@ -127,11 +133,11 @@ depend only on the commit.
pnpm test:package-release # scripts/test-package-release.sh
```
1. **Builds the train** from this checkout.
1. **Builds the train** from this checkout. Nothing is built before it.
2. **Builds it again** from a second worktree of the same commit at another absolute path, and
requires both release directories to be byte-identical: every tarball, `SHA256SUMS`, and the
manifest.
3. **Runs the standalone consumer proof** (`scripts/test-standalone-module.sh --release <dir>`) on
3. **Builds Core's verifier**, then **runs the standalone consumer proof** (`scripts/test-standalone-module.sh --release <dir>`) on
that release:
- **The release is checked first.** Every artifact must match the manifest's size, SHA-256, and
integrity, and the release must not be built from a modified tree.
@@ -139,7 +145,8 @@ pnpm test:package-release # scripts/test-package-release.sh
`tests/fixtures/standalone-module`, copied outside the repository. Every `@ekklesia/*`
package, direct or transitive, resolves to its release tarball. There is no workspace link and
no Ekklesia source checkout. The install runs offline: third-party packages come from the
local pnpm store, so no registry is involved at all.
workspace install's own pnpm store, named explicitly because pnpm otherwise picks a store per
file system, so no registry is involved at all.
- **The lockfile matches the release.** It must record each artifact's own integrity. The module
then reinstalls from the lockfile alone with `--frozen-lockfile`.
- **The install has the right shape.** There is no `drizzle-orm`, and exactly one `react`, which
+1 -1
View File
@@ -42,7 +42,7 @@
"test": "turbo run test test:root",
"test:migrations:integration": "pnpm build && bash scripts/test-migrations-integration.sh",
"test:root": "vitest run --config vitest.config.ts",
"test:package-release": "pnpm build && bash scripts/test-package-release.sh",
"test:package-release": "bash scripts/test-package-release.sh",
"test:watch": "vitest",
"typecheck": "turbo run typecheck typecheck:root",
"typecheck:root": "tsc --noEmit --project tsconfig.json",
+26 -4
View File
@@ -433,6 +433,20 @@ async function inspectTarball(
return [...new Set(problems)].map((problem) => `${entry.name} ${problem}.`);
}
/**
* Every uncommitted change in the source tree, as `git status --short` lines followed by
* `git diff --stat`, all relative to the repository root; empty for a clean tree. Ignored files
* (build output, `node_modules`) are not source and never count.
*/
async function sourceTreeChanges(): Promise<string[]> {
const status = (
await run("git", ["status", "--short", "--untracked-files=all"], repositoryRoot)
).trimEnd();
if (status === "") return [];
const stat = (await run("git", ["diff", "--stat"], repositoryRoot)).trimEnd();
return [...status.split("\n"), ...(stat === "" ? [] : stat.split("\n"))];
}
export class PackageReleaseError extends Error {
readonly problems: readonly string[];
@@ -462,13 +476,12 @@ export async function buildPackageRelease(
if (setProblems.length > 0) throw new PackageReleaseError(setProblems);
const revision = (await run("git", ["rev-parse", "HEAD"], repositoryRoot)).trim();
const modified =
(
await run("git", ["status", "--porcelain", "--untracked-files=all"], repositoryRoot)
).trim() !== "";
const dirty = await sourceTreeChanges();
const modified = dirty.length > 0;
if (modified && options.allowModified !== true) {
throw new PackageReleaseError([
"The source tree has uncommitted changes. A release is built from a clean commit.",
...dirty,
]);
}
@@ -498,6 +511,15 @@ export async function buildPackageRelease(
],
repositoryRoot,
);
// The source was clean before the build; it must still be, so what is packed is the commit. A build
// that wrote a tracked file, or an untracked file Git does not ignore, fails the release here.
const changedByBuild = (await sourceTreeChanges()).filter((line) => !dirty.includes(line));
if (changedByBuild.length > 0) {
throw new PackageReleaseError([
"The build changed the source tree. A release packs exactly its commit.",
...changedByBuild,
]);
}
const scratch = await mkdtemp(path.join(tmpdir(), "ekklesia-package-release-"));
try {
+16 -3
View File
@@ -9,8 +9,11 @@
# 2. Builds it again from a separate worktree of the same commit, at another absolute path, and
# requires the two release directories to be byte-identical: every tarball, SHA256SUMS, and the
# manifest.
# 3. Installs the release into the standalone fixture and proves it builds, packages, and verifies
# (`scripts/test-standalone-module.sh`).
# 3. Builds Core's verifier, then installs the release into the standalone fixture and proves it
# builds, packages, and verifies (`scripts/test-standalone-module.sh`).
#
# Nothing is built before step 1: the release builder asserts a clean source tree first, builds, and
# asserts the build changed none of it. Every offline install uses the workspace install's own store.
#
# `--allow-modified` is for local development on a modified source tree. The second build is then
# made in this checkout again, because a worktree of the commit would not hold the modifications.
@@ -25,6 +28,11 @@ elif (($#)); then
exit 64
fi
# The workspace install's own pnpm store. pnpm chooses a store per file system, so an install in a
# temporary directory would otherwise look in another, empty one; every install here is offline.
store="$(dirname "$(cd "$repository_root" && pnpm store path)")"
export EKKLESIA_PNPM_STORE="$store"
work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-package-release-XXXXXX")"
worktree="$work/second-checkout"
cleanup() {
@@ -44,7 +52,7 @@ if ((${#allow_modified[@]})); then
node "$repository_root/scripts/package-release.ts" --out "$work/second" "${allow_modified[@]}" >/dev/null
else
git -C "$repository_root" worktree add --detach --quiet "$worktree" HEAD
(cd "$worktree" && pnpm install --offline --frozen-lockfile --reporter=silent)
(cd "$worktree" && pnpm install --offline --frozen-lockfile --store-dir "$store" --reporter=silent)
node "$worktree/scripts/package-release.ts" --out "$work/second" >/dev/null
fi
if ! diff -r "$release" "$(find "$work/second" -mindepth 1 -maxdepth 1 -type d)"; then
@@ -53,6 +61,11 @@ if ! diff -r "$release" "$(find "$work/second" -mindepth 1 -maxdepth 1 -type d)"
fi
echo " byte-identical: $(ls "$release" | wc -l) files"
# The consumer proof verifies with Core's own verifier. Built only now, after both releases: the
# release builder asserts a clean source tree before it builds anything.
echo "==> Building Core's verifier"
(cd "$repository_root" && pnpm exec turbo run build --filter=@ekklesia/core --output-logs=errors-only >/dev/null)
echo "==> Consuming the release from a standalone module"
bash "$repository_root/scripts/test-standalone-module.sh" --release "$release" "${allow_modified[@]}"
+7 -2
View File
@@ -12,6 +12,7 @@
# UI, packages with the release's own `@ekklesia/module-pack`, and verifies with Core's verifier.
#
# `--allow-modified` accepts a release built from a modified source tree, for local development.
# Core must be built (`core/dist`): the module is verified with Core's own verifier.
set -euo pipefail
repository_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
@@ -37,6 +38,10 @@ while (($#)); do
done
[[ -n "$release" && -d "$release" ]] || fail "no release directory given"
# Third-party packages come from the workspace install's own pnpm store, offline. pnpm chooses a
# store per file system, so the module, in a temporary directory, is pointed at it explicitly.
store="${EKKLESIA_PNPM_STORE:-$(dirname "$(cd "$repository_root" && pnpm store path)")}"
work="$(mktemp -d "${TMPDIR:-/tmp}/ekklesia-standalone-XXXXXX")"
trap 'rm -rf "$work"' EXIT
@@ -96,7 +101,7 @@ pinned="$(node -p 'require(process.argv[1]).devDependencies["@ekklesia/module-pa
} >"$module/pnpm-workspace.yaml"
echo "==> Installing the module from the release tarballs, offline"
(cd "$module" && pnpm install --offline --reporter=append-only >"$work/install.log" 2>&1) ||
(cd "$module" && pnpm install --offline --store-dir "$store" --reporter=append-only >"$work/install.log" 2>&1) ||
{
tail -40 "$work/install.log" >&2
fail "install (third-party packages come from the local store: run pnpm install in the workspace first)"
@@ -125,7 +130,7 @@ EOF
echo "==> Reinstalling from the lockfile alone (--frozen-lockfile)"
rm -rf "$module/node_modules"
(cd "$module" && pnpm install --offline --frozen-lockfile --reporter=append-only >"$work/frozen.log" 2>&1) ||
(cd "$module" && pnpm install --offline --frozen-lockfile --store-dir "$store" --reporter=append-only >"$work/frozen.log" 2>&1) ||
{ tail -40 "$work/frozen.log" >&2; fail "frozen-lockfile install"; }
echo "==> Checking what the install resolved"
+12
View File
@@ -1,3 +1,4 @@
import { spawnSync } from "node:child_process";
import { readdir, readFile } from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
@@ -361,6 +362,17 @@ describe("the pack hook", () => {
});
});
describe("a clean source tree", () => {
it("ignores pnpm's package store, which pnpm creates in a checkout that is its own mount", () => {
// In CI the checkout is a workspace volume, so `pnpm install` puts `.pnpm-store/` at its root:
// a package cache, never source, which must not make the release builder's clean tree dirty.
const ignored = spawnSync("git", ["check-ignore", "--quiet", ".pnpm-store/v11/index.db"], {
cwd: repositoryRoot,
});
expect(ignored.status).toBe(0);
});
});
describe("the release directory", () => {
const version = "0.1.2";
function release(mutate?: (files: Map<string, Uint8Array>) => void) {