diff --git a/docs/architecture/decisions/0047-module-ci-harness.md b/docs/architecture/decisions/0047-module-ci-harness.md new file mode 100644 index 0000000..98b1d53 --- /dev/null +++ b/docs/architecture/decisions/0047-module-ci-harness.md @@ -0,0 +1,238 @@ +# 0047: Every module proves itself inside Ekklesia, with one harness the platform publishes + +- Status: Proposed +- Date: 2026-10-05 +- Supersedes: none +- Related: [ADR 0016](0016-module-database-access.md), + [ADR 0021](0021-modules-as-hostable-products.md), + [ADR 0028](0028-module-operational-lifecycle.md), + [ADR 0030](0030-host-provided-module-object-storage.md), + [ADR 0033](0033-display-kiosk-host-contract.md), + [ADR 0036](0036-managed-device-identity-and-pairing.md), + [ADR 0041](0041-module-browser-contributions.md), + [ADR 0043](0043-display-offline-code-fallback.md), + [ADR 0044](0044-self-hosted-reference-deployment.md), + [ADR 0045](0045-split-dashboard-and-api-origins.md) + +## Context + +Modules are independently versioned products (ADR 0021). They live in their own repositories, and +none of them can be built, run, or rendered without Ekklesia. The first production module, Signage, +reached a church with three defects that only show when it runs **inside** Ekklesia, in a real +browser: + +- every heartbeat its display sent was refused with a 400, because the display and the server + disagreed about `null`; +- a display that failed one request at start-up never recovered without a reload; +- its media cache leaked one blob per slide per refresh, until the browser refused the display's own + requests. + +Signage had unit tests, browser tests against a stubbed host, and no CI. Every test passed, because +each half was tested against its own idea of the other, and nothing ever ran the module in the +platform it ships into. + +The platform's own deployment smoke (ADR 0044, ADR 0045) does run a module in a real browser, but +only the repository's template, and only far enough to mount it. The instance repository runs the +smoke against its real modules, but only at release time, after a module has already been tagged. +Each module repository that wants more than that has to assemble the platform, build an image, +start a deployment, and drive a browser itself. Signage is about to do exactly that by hand. The +second module, Classroom, would copy it, and the copies would drift from the platform they test. + +## Decision + +### The platform publishes a versioned module harness + +Each platform release publishes one OCI image, `ekklesia-module-harness:`. It is +built from the platform at that release tag by the platform's own release CI, and pushed to the +organization's registry. A version is never overwritten. Modules reference it **by digest**. + +The harness contains the platform source at that version: its `deploy/` directory, its Dockerfile, +its lockfile, and its smoke scripts. It also contains the pinned Node.js and pnpm, the browser +driver, and the pinned headless Chromium image reference. Given a module checkout, it: + +1. **assembles** the module into the platform at `modules/`, exactly as an instance does (ADR + 0044), and installs with `--frozen-lockfile`; +2. runs the platform's own gates against that set: `pnpm boundaries`, `modules:validate`, + `migrations:validate`, and the build; +3. **builds** the deployment image from the assembled workspace with the platform's Dockerfile; +4. starts the **reference deployment** in the split topology (ADR 0045), with a disposable + PostgreSQL and object store, Caddy's internal CA, and freshly generated secrets; +5. plans, then **explicitly applies**, migrations against that fresh database. This is the only + database the harness ever migrates; +6. creates the initial owner, grants the module's declared permissions to a test person, and + **mounts every UI surface the manifest declares**, in headless Chromium. That covers: + - the primary UI in the Dashboard; + - the display, paired as a device with no human session (ADR 0036); + - each browser contribution, whenever the module that hosts it is present (ADR 0041); +7. runs a short **display soak**, and the **no-dead-ends** injections below; +8. tears everything down: containers, networks, volumes, and the work directory. + +It **fails** on any of the following: + +- a `console.error` or uncaught exception on any page; +- an unhandled promise rejection; +- any 4xx or 5xx response that the harness did not itself expect. Expected ones are listed in the + harness, for example a display's descriptor before it is paired; +- a page that never mounts; +- a failed or refused request (CORS, blocked, network error); +- a soak leak. + +Each failure line names the surface, the request path (never its query), and the step. + +### The display soak and the no-dead-ends checks are generic + +These checks come from what Signage got wrong, and they apply to any module with a display. + +- **Soak**, two minutes by default. The display stays mounted with no remount and no offline + indicator. Its realtime streams stay healthy. Live object URLs stay bounded. JavaScript heap + growth, measured over DevTools, stays within a fixed ratio. +- **No dead ends.** One at a time, the harness refuses the first request to each module route the + display calls at start-up, and the first realtime connection. Within a bounded time and with no + reload, the display must make that request again and succeed. A display has nobody to press + Retry. + +Module-specific behaviour, such as Signage's heartbeat being recorded, is not guessed by the +harness. A module declares it (below). + +### A module is tested against both ends of its compatibility range + +A module's CI runs the harness twice: against the **lowest** and the **newest** published platform +version within its manifest's `compatibility.platform` range. Testing every version in between adds +cost and finds little. Testing only the newest misses the floor the manifest promises to work on. + +The two versions, and their harness digests, are pinned in the module's repository. The harness +refuses to run if: + +- a pinned version lies outside the module's range; +- the lowest pinned version is not the range's floor. + +Moving to a newer platform is a deliberate, reviewed pin change, as an instance's is (ADR 0044). It +is never a moving tag. + +A frozen install of the module assembled into each of those two platforms needs a lockfile per +pinned version. The module keeps them (`ci/lock/.yaml`), and the harness provides +the one command that regenerates them inside its own pinned Node.js. A stale lockfile fails the +frozen install, which is the signal to regenerate it in a reviewed change. + +### Modules declare extra smoke steps, beside their manifest rather than in it + +A module may add steps of its own in `ekklesia.harness.json`, next to `ekklesia.module.json`. Steps +are declarative first: + +- open a surface; +- wait for a test id; +- call one of the module's routes as a named role, expecting a status; +- soak for N minutes; +- inject a failure into a named route; +- check a value in the module's own table, through its own route. + +For what a declaration cannot express, a step may name a driver script in the module's repository. +The harness runs that script with the same DevTools client and the same failure rules. + +**This is deliberately not a manifest field.** Manifest versions are strict, hard cutovers that every +host validates (ADR 0033). A test step is not a runtime contract, and a host must never read one. +Putting steps in the manifest would force a manifest version bump on every module for something +that has no meaning outside CI. + +### Later: the harness checks what a module actually touches against what it declares + +In a later phase, the harness records what the module does while it runs: + +- the tables and schemas it reads and writes; +- the buckets it uses; +- its outbound network connections, with the egress network blocked and every attempt logged. + +It compares them with what the manifest declares (ADR 0016, ADR 0030). A module that touches what it +did not declare fails. This is the module-trust check. The egress half needs a manifest declaration +of outbound destinations that does not exist yet, and is a separate decision. This ADR fixes only +the harness's shape, so that the recording phases can be added without changing how modules call +it. + +### The template's workflow stays short, and calls the harness by digest + +`templates/module` ships a workflow that: + +- checks out the module with `persist-credentials: false` and pinned actions; +- runs the module's own unit tests, typecheck, and lint; +- runs the harness, once per pinned platform version, with one `docker run` of the harness image + **by digest**. The module directory is mounted read-only, and the runner's Docker is used exactly + as the platform's smoke uses it. + +It references no secret, needs no registry credential (the harness image is public within the +organization), and uses no `pull_request_target`. The trusted-code-only rule for self-hosted runners +applies to module repositories exactly as it does to the platform (see `docs/development/security.md`). + +**Why `docker run`, and not a reusable workflow or a composite action.** The inventoried server is +Gitea 1.24.2. + +- Its runner does support calling a reusable workflow from another repository, by cloning that + repository. That support is thinly documented, and has known problems in practice. +- Actions and reusable workflows from **private** repositories only arrive with Gitea 1.26 + (go-gitea/gitea#32562). +- Composite actions from another repository work through absolute `uses:` URLs. That would still + wrap the same `docker run`, adding a second pinned reference and nothing else. + +An image digest is the one reference that is equally immutable, equally reviewable, and works +unchanged on Gitea 1.24, Gitea 1.26, and GitHub Actions. A composite action can be added later as a +convenience over the same image, without changing what a module pins. + +### Publishing follows the instance's release pattern + +The harness image is pushed only by a `publish` job that runs on `v*` tags, after the platform's CI +has built and tested that exact image (checked by image ID). That job: + +- uses a write-only package token, the only secret in the platform's workflow; +- refuses to overwrite an existing version; +- prints the digest. + +Pull requests and branch builds build the harness and run it against the template, but never push. +The registry is reached exactly as the instance's publish job reaches it. + +### Adoption + +- **Signage** uses an interim CI job until the first harness is published. That job assembles + Signage into a pinned platform commit and runs the platform's display-device smoke scenarios. + Signage switches to the harness as soon as one is published. +- **Classroom** follows. +- **The template** carries the workflow from the first harness release on, so every new module has + it from its first commit. + +## Consequences + +- A module cannot reach a release without having run inside Ekklesia, in a real browser, at both + ends of the platform range it claims. +- The defects that reached the first production Signage deployment fail a module's CI instead: + refused requests, dead ends, leaks, and console errors. +- The platform owns one harness, versioned with itself. Modules do not copy deployment scripts that + then drift. +- Every module's CI needs Docker on its runner, and takes minutes rather than seconds. The harness + runs as separate jobs from the unit tests, so a quick failure still reports quickly. +- The platform's release process gains one published artifact, one credential, and one more failure + mode at release time. +- A module now carries a lockfile for each pinned platform version. The harness makes regenerating + them one command, and a stale one fails visibly. + +## Alternatives + +- **Each module keeps its own copy of the smoke scripts.** Rejected. That is today's situation in + embryo, and it drifts. +- **A reusable workflow in the platform repository.** Rejected for now, for the Gitea 1.24 reasons + above. It would also give the harness's behaviour to whichever runner and server version happens + to expand it. +- **Publish the harness as an npm package.** Rejected. The harness needs Docker, a browser, and the + platform's deployment files anyway, so a package would ship a fraction of it and assume the rest. +- **Test only the newest platform version.** Rejected. A module's range floor would go untested + until an installation that has not upgraded meets it. +- **Put smoke steps in the manifest.** Rejected. Manifest versions are hard cutovers with runtime + meaning, and test steps have neither. +- **Run the harness in the platform's CI against every known module.** Rejected. The platform would + need every module's source and would fail on modules it does not own. The direction is module + tests platform, not the reverse. + +## Deferred + +- The data-access and egress recording phases, and the manifest declaration egress needs. +- A composite action over the harness image. +- Running the harness against a module's declared dependencies at their own pinned versions, for + contributions hosted by another module. +- Visual regression of mounted surfaces. diff --git a/docs/architecture/decisions/README.md b/docs/architecture/decisions/README.md index 8e3d740..b8b9858 100644 --- a/docs/architecture/decisions/README.md +++ b/docs/architecture/decisions/README.md @@ -55,3 +55,4 @@ ADRs capture durable choices that affect more than one package or phase. - [0044: Ekklesia ships one self-hosted reference deployment, composed from pinned sources](0044-self-hosted-reference-deployment.md) — Accepted - [0045: The Dashboard and the API may live on separate origins](0045-split-dashboard-and-api-origins.md) — Accepted - [0046: Ekklesia's branding is never forced on an installation](0046-installation-owned-branding.md) — Proposed +- [0047: Every module proves itself inside Ekklesia, with one harness the platform publishes](0047-module-ci-harness.md) — Proposed