Ekklesia/docs/development/releases.md

# Releases and the public package train

An Ekklesia platform release is one version. The platform image, the private workspace packages,
and the public `@ekklesia/*` module-development packages all carry it. This page covers the order a
release happens in and the public package train
([ADR 0053](../architecture/decisions/0053-standalone-module-development-packages.md)): how its
artifacts are produced, what proves them, and how exactly those artifacts are published.

> **The packages are publicly readable, on purpose**
> ([ADR 0054](../architecture/decisions/0054-public-package-reads-before-launch.md)). Anyone can
> search, read, and install `@ekklesia/*` from the registry with no account and no token, like any
> public npm package. That is not a public launch: Ekklesia's source is already publicly readable,
> and the project is simply not promoted yet. Only **publishing** is restricted, to one manual,
> authenticated workflow. Before 1.0, the APIs may change between versions under the project's
> SemVer policy: a 0.x minor version may break them.

## Where each step sits

```text
version set ─▶ build ─▶ test ─▶ pack the public package train ─▶ standalone consumer proof
                                                                          │
     tag and release (maintainer) ◀─ publish the exact train (manual) ◀───┘
```

1. **Version set.** A `chore(release): prepare X` pull request sets the release version: the root
   `package.json`, every platform workspace package (the module template versions as a module, not
   with the platform), the image build example in `deploy/README.md`, and the standalone fixture's
   exact `@ekklesia/module-pack` pin and `@ekklesia/*` ranges
   (`tests/fixtures/standalone-module/package.json`). The tag must point at a commit that carries
   the version it names.

   A minor version (0.x → 0.(x+1)) is a breaking release before 1.0, so manifests that declare
   `compatibility.platform` and `compatibility.moduleSdk` as `^0.x.0` no longer match it. In the
   same pull request, move to the new range every module built for this release (the template, the
   standalone fixture, the native proof module), the test fixtures that model a module of the
   current release or the current platform, and the module-development guide's examples. A patch
   version changes none of these.
2. **Build and test.** `pnpm verify`, the integration suite, and the deploy smoke, as for any change.
3. **Pack the public package train.** `pnpm packages:release` (below).
4. **Standalone consumer proof.** A module outside the repository consumes exactly those
   artifacts (below).
5. **Publish the exact train.** By hand, with the `Publish packages` workflow on the version-set
   commit on `main` ([Publishing the train](#publishing-the-train)). It publishes the artifacts its
   own run's gate (steps 3 and 4) proved, and nothing else.
6. **Tag and release.** Unchanged: the maintainer tags the merge commit of the version-set pull
   request. Nothing here creates, moves, or requires a tag.

Steps 2 to 4 run in CI on every pull request and on `main`, and CI also publishes each proven train
to a disposable registry (`package-publication`), never the real one. Step 5 runs only when a
maintainer starts it.

## The public package train

| Package | Role |
| --- | --- |
| `@ekklesia/module-sdk`, `@ekklesia/module-database`, `@ekklesia/module-ui-sdk`, `@ekklesia/design-tokens` | author API |
| `@ekklesia/module-ui-build`, `@ekklesia/module-pack`, `@ekklesia/typescript-config` | tooling |
| `@ekklesia/ui` | publishable, pre-1.0 component API |
| `@ekklesia/contracts` | implementation dependency of the packages above: published only because they need it, never a supported direct dependency of a module |

The list lives in `scripts/package-release.ts` (`PUBLIC_PACKAGES`). Every other workspace package is
`private`, and the release builder refuses to run if a package outside the list is publishable or
carries the `module-development` distribution marker.

- **One version.** Every package in the train has the platform release version.
- **One train.** Every `@ekklesia/*` dependency in a packed manifest names a package of the same
  train at exactly that version. `workspace:` ranges become those exact versions when packed.
- **No independent versions.** A package cannot be released on its own.

## Producing the release artifacts

```bash
pnpm packages:release          # writes .package-release/ekklesia-packages-<version>/
```

`scripts/package-release.ts`:

1. **Refuses a modified source tree, before it builds anything.** A release is built from a
   commit. A tree is clean when `git status` shows no change to a tracked file and no untracked file
   that Git does not ignore. Ignored files are build output and package-manager state:
   `node_modules`, `dist`, `.turbo`, and pnpm's `.pnpm-store`, which pnpm creates at the checkout's
   root when the checkout is its own mount (a CI runner's workspace volume). The failure lists every
   change. `--allow-modified` exists for local development only; its manifest then says
   `"modified": true`, and such a release must never be published.
2. **Checks the release set and versions** in the workspace.
3. **Rebuilds from clean output.** It removes each public package's `dist/` and rebuilds without the
   build cache, so output from a deleted source file can never ship. It then checks the source tree
   again: a build that changed a tracked file, or wrote an untracked one Git does not ignore, fails
   the release, because what is packed must be exactly the commit.
4. **Packs each package with `pnpm pack`.** Pack-only pnpm hooks (`scripts/lib/pack-hooks.cjs`)
   apply, never the workspace's install settings ([Reproducibility](#reproducibility)).
5. **Inspects every tarball.** It checks:
   - the expected name, version, and file name;
   - a `LICENSE`, and only the `files` the manifest declares, each built file with the source it
     came from, and no source, test, or configuration file;
   - no `workspace:` range, no `devDependencies`, no `publishConfig`, and no install-time lifecycle
     script;
   - only public `@ekklesia/*` dependencies, at the train's version;
   - the distribution marker;
   - no local path (the checkout, the home directory, or the temporary directory) in any file.
6. **Checks the set.** The release must hold exactly the expected tarballs: none missing, none extra.
7. **Writes the release directory only when everything passed,** in one step. It refuses to
   overwrite one that exists.

```text
ekklesia-packages-<version>/
├── ekklesia-contracts-<version>.tgz
├── ekklesia-design-tokens-<version>.tgz
├── … one .tgz per package in the train
├── SHA256SUMS                     `sha256sum -c SHA256SUMS` compatible
└── ekklesia-package-train.json    the release manifest
```

### The release manifest

`ekklesia-package-train.json` describes the train by content. It is provider-neutral: it names no
registry and is not a registry catalog.

```json
{
  "packageTrain": 1,
  "version": "0.2.0",
  "source": { "modified": false, "revision": "<full commit id>" },
  "packages": [
    {
      "file": "ekklesia-contracts-0.2.0.tgz",
      "integrity": "sha512-<base64>",
      "name": "@ekklesia/contracts",
      "role": "implementation",
      "sha256": "<hex>",
      "size": 87379,
      "version": "0.2.0"
    }
  ]
}
```

`integrity` is the npm `dist.integrity` form: a registry computes the same value from the same
bytes, and a module's lockfile records it. The packages are sorted by name, and the file's bytes
depend only on the commit.

## Proving the artifacts

```bash
pnpm test:package-release      # scripts/test-package-release.sh
```

1. **Builds the train** from this checkout. Nothing is built before it.
2. **Builds it again** from a second worktree of the same commit at another absolute path, and
   requires both release directories to be byte-identical: every tarball, `SHA256SUMS`, and the
   manifest.
3. **Builds Core's verifier**, then **runs the standalone consumer proof** (`scripts/test-standalone-module.sh --release <dir>`) on
   that release:
   - **The release is checked first.** Every artifact must match the manifest's size, SHA-256, and
     integrity, and the release must not be built from a modified tree.
   - **The fixture module installs from those tarballs alone.** It is
     `tests/fixtures/standalone-module`, copied outside the repository. Every `@ekklesia/*`
     package, direct or transitive, resolves to its release tarball. There is no workspace link and
     no Ekklesia source checkout. The install runs offline: third-party packages come from the
     workspace install's own pnpm store, named explicitly because pnpm otherwise picks a store per
     file system, so no registry is involved at all.
   - **The lockfile matches the release.** It must record each artifact's own integrity. The module
     then reinstalls from the lockfile alone with `--frozen-lockfile`.
   - **The install has the right shape.** There is no `drizzle-orm`, and exactly one `react`, which
     `@ekklesia/ui` resolves to as well. `@ekklesia/contracts` is installed only transitively: it is
     not in the module's `package.json` and not directly under `node_modules/@ekklesia`. Every
     installed `@ekklesia/*` package, `module-pack` included, is exactly the release version, and
     the fixture pins `module-pack` exactly.
   - **The module builds.** It typechecks with `skipLibCheck: false`, so every published declaration
     it reaches is checked, then builds its backend and UI.
   - **It packages and verifies.** It is packed with the release's own `ekklesia-pack-module`, with
     no workspace file present. Core's verifier then verifies it. The package must:
     - incorporate the train's packages at the release version;
     - name the release's module-pack as its packager;
     - credit no Ekklesia package and exactly one React;
     - contain no local path.

## Reproducibility

Two builds of the same commit produce byte-identical tarballs, and CI proves it on every run.

- **pnpm normalizes the tar entries.** Every entry gets the fixed modification time 1985-10-26, owner
  0, and a mode of 0644, or 0755 when executable. The gzip header records no time.
- **The build output is deterministic,** and source maps use relative paths.
- **One nondeterminism came from pnpm itself.** pnpm rewrites `workspace:` ranges concurrently and
  inserts each dependency as its lookup resolves. A package with more than one `workspace:`
  dependency in one field could therefore be packed with those keys in a different order. Builds
  from two checkouts of one commit differed in exactly that way: `@ekklesia/module-pack`'s
  `package.json` ordered `@ekklesia/module-sdk` and `@ekklesia/module-database` differently.
  - **The fix is pnpm's own hook.** The pack-only `beforePacking` hook puts each dependency map back
    in the source `package.json`'s key order. It changes no name, range, or field. The tarball stays
    an ordinary `pnpm pack` tarball.
  - **The inspection enforces it.** It fails any packed map whose order differs from its source.
- **The hook also drops `devDependencies`** from packed manifests. A consumer never installs them,
  and a tooling package's test-only development dependencies named private workspace packages
  outside the train.

**What reproducibility does not cover:**
- The module package the consumer proof builds is deterministic only for a fixed install layout.
  pnpm names a `file:` tarball's install directory after its path, and bundlers keep those names in
  comments. So the proof always installs from the same relative path. From a registry, the directory
  is simply `<name>@<version>`.
- Byte identity is proven for the pinned toolchain: Node.js from `.node-version`, and pnpm from
  `packageManager`. Another Node.js or zlib build may compress differently. Each artifact's
  integrity is recorded either way, and that, not byte identity, is what a publication step must
  verify.

## Publishing the train

The registry is the Gitea npm registry under the `Zayin` owner:

```text
https://git.zayin.net/api/packages/Zayin/npm/
```

| Who | May |
| --- | --- |
| Anyone, signed in or not | search, read metadata, download, and install `@ekklesia/*` |
| The publisher account, through the publish workflow | publish new versions and move dist-tags |
| Anyone else | nothing more: publishing needs the publisher team's package write permission |

### The workflow

`.github/workflows/publish-packages.yml` (Actions → Publish packages → Run workflow, on `main`,
with the release version). Nothing else can start it: no pull request or push does, and CI holds no
registry credential at all.

1. **The gate.** `package-release` runs the same credential-free gate as CI (build, reproduce,
   consume) and hands the proven release on as a workflow artifact, with its manifest's SHA-256.
2. **The publish job.** It builds and installs nothing. It downloads the artifact and runs
   `scripts/publish-package-train.ts` with the secret `EKKLESIA_PACKAGES_PUBLISH_TOKEN`, which only
   this step receives.
3. **The publication record** is kept as an artifact of the run.

### What the publication tool does

`scripts/publish-package-train.ts` publishes exactly the `.tgz` files in the release directory. It
never builds, packs, or edits a package, and it never deletes, unpublishes, or overwrites anything.

1. **Validates the whole train before sending anything.** The manifest must be the one the gate
   proved (`--manifest-sha256`), for the requested version and this commit (`--revision`), from a
   clean tree (`"modified": false`), with exactly the nine expected packages. Every artifact's file
   name, size, SHA-256, integrity, `SHA256SUMS` line, and the `package.json` inside it must match.
2. **Publishes in dependency order.** Each package comes after the train packages it depends on,
   then alphabetically. For each package:
   - **absent:** `npm publish <file.tgz>` under the staging dist-tag `ekklesia-staging`, given the
     artifact's absolute path (npm runs in a temporary directory of its own), after the file is
     checked once more against the manifest. npm sends the file's own bytes and runs no script for a
     file; the registry checks them against their integrity and stores them as they are;
   - **present with the same bytes:** accepted as already published, and verified;
   - **present with other bytes:** a hard stop. Nothing is deleted or overwritten, nothing after it
     is published, and the version is burned ([Immutability](#immutability)).
3. **Reads every package back.** The registry's metadata must name the package and version, report
   the artifact's integrity, and list the same `dependencies`, `peerDependencies`,
   `optionalDependencies`, and `bin` as the tarball's `package.json` (npm writes `bin` paths without
   a leading `./`). Its tarball URL must be on the configured registry, and the tarball it serves must
   be the release artifact, byte for byte. With `--require-public-read`, an anonymous download must
   give the same bytes too. A package is published only once all of that holds.
4. **Moves `latest` last.** Only when all nine are verified does `latest` move to the train, never
   backwards past a newer version. The train is published when every package's `latest` agrees.
5. **Writes the publication record** (`--record`).

A rerun is always safe. It publishes only what is missing, accepts what is already there and
identical, and moves only the tags that are not yet right. A train interrupted after five packages,
or after four tags, finishes on the next run. No retry ever deletes anything.

### The publication record

Evidence that one train was published, not a second package catalog, and never a secret:

```json
{
  "publicationRecord": 1,
  "packageTrain": { "version": "0.2.0", "revision": "<commit>", "manifestSha256": "<hex>" },
  "registry": "https://git.zayin.net/api/packages/Zayin/npm/",
  "finalTag": "latest",
  "completedAt": "<ISO time>",
  "packages": [
    {
      "name": "@ekklesia/contracts",
      "version": "0.2.0",
      "file": "ekklesia-contracts-0.2.0.tgz",
      "outcome": "published",
      "artifact": { "sha256": "<hex>", "integrity": "sha512-…", "size": 87379 },
      "registry": { "tarball": "<url>", "sha256": "<hex>", "integrity": "sha512-…", "publicRead": true },
      "finalTag": "moved",
      "distTags": { "ekklesia-staging": "0.2.0", "latest": "0.2.0" }
    }
  ]
}
```

`outcome` is `published` or `already-present`. `finalTag` is `moved`, `already-current`, or
`kept-newer` (a newer version already holds `latest`).

### Credentials

- **One credential:** `EKKLESIA_PACKAGES_PUBLISH_TOKEN`, a repository Actions secret, used only by
  the publish step. Nobody needs a token to read or install.
- **The publisher** is a dedicated account, member only of a `Zayin` team whose one
  permission is Packages: Write. Its token has only the `write:package` scope. Gitea's
  `write:package` also allows deleting packages; the tool never does, and nothing else uses the
  token.
- **No token in git or in an `.npmrc`.** The tool writes a temporary npm configuration that refers to
  the variable (`_authToken=${EKKLESIA_PACKAGES_PUBLISH_TOKEN}`), ignores every user, global, and
  project npm configuration, sends the token only to the registry itself, and removes the
  configuration afterwards.
- **What remains trusted:** Gitea gives repository secrets to workflows run for pull requests from
  branches of this same repository, so anyone who can push a branch here could read the secret
  through a changed workflow. Keep write access to this repository to the maintainers.

### Immutability

Gitea refuses to upload a version that exists (409). But it lets anyone with package write
permission delete a version, deletion leaves nothing behind, and an owner's package cleanup rules
can delete versions on a schedule. So immutability is Ekklesia's policy, not the registry's
guarantee:

- **A published version is never deleted,** by anyone, for any reason short of a security incident.
- **A version number is never reused with other bytes.** A version found on the registry with other
  bytes is burned: investigate how it got there, and publish the train under a new version.
- **No package cleanup rule** may be configured for the `Zayin` owner.
- **The publisher never deletes.** The tool has no delete, unpublish, or overwrite, and its tests
  check that.

Module lockfiles record each package's integrity, so a replaced package would fail every install of
it, rather than install silently.

### Moving to another registry

Only configuration changes: the registry URL (the workflow's `EKKLESIA_PACKAGES_REGISTRY`), the
publisher credential, and modules' `@ekklesia:registry`. Modules then regenerate their lockfiles,
because Gitea's tarball URLs are its own. Package sources, the train, its manifests, and every
artifact already built stay as they are; the same bytes keep the same integrity.

### Proving the publication machinery

`pnpm test:package-publication` (`scripts/test-package-publication.sh`, the CI `package-publication`
job) publishes a real release train to a disposable Gitea 1.27.2 configured like the real one:
public owners, a publisher team, and an outsider. In CI it publishes the very artifact the
`package-release` job handed on, after checking its manifest hash. It proves:

- anonymous search, metadata, and download work;
- publishing without credentials, or by a user outside the publisher team, fails;
- the round trip is byte-identical, also anonymously;
- the registry refuses a second upload of a published version;
- a train interrupted before its sixth package resumes, verifying the five and publishing the rest,
  with `latest` untouched until all nine are verified;
- a version already published with other bytes stops the train, deleting and overwriting nothing,
  and publishing nothing after it, on every retry;
- a train interrupted while moving `latest` converges on a rerun, moving only the missing tags.

### The first publication

The first train published is **0.2.0**, the next platform release, not 0.1.2. The `v0.1.2` tag
predates the curated SDK and this release tooling, so packages built now and called 0.1.2 would
claim to be source that 0.1.2 never contained.

Before it, an administrator of the Gitea instance:

1. creates the dedicated publisher account;
2. creates a `Zayin` team, for example `package-publishers`, with only the Packages
   unit at Write, and adds the account to it;
3. creates a token for the account with only `write:package`, and stores it as the repository
   Actions secret `EKKLESIA_PACKAGES_PUBLISH_TOKEN`;
4. checks that the owner has no package cleanup rule.

Then the release proceeds as usual: the version-set pull request merges, a maintainer starts
Publish packages on `main` with that version, and tags the commit once the train is published.