docs: propose ADR 0047, a published harness every module runs in CI

Signage reached production with three defects that appear only inside Ekklesia in a real browser, and no CI that ran it there. The ADR proposes: - a versioned harness image, ekklesia-module-harness:<platform version>, built and pushed by the platform's release CI; - what the harness does: assemble a module into that platform, validate and build it, migrate a disposable database, start the split reference deployment, and mount every declared UI surface in headless Chromium; - what fails it: console errors, unexpected 4xx/5xx, unhandled rejections, a display soak, and generic no-dead-ends injections; - testing at the lowest and newest platform versions of a module's range; - extra smoke steps in ekklesia.harness.json, beside the manifest rather than in it; - a later data-access and egress check against declarations; - a short template workflow that calls the harness by digest with one `docker run`. Not a reusable workflow: thinly supported on Gitea 1.24, and private sources only arrive in 1.26. Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-05 19:07:55 -05:00
2 changed files with 239 additions and 0 deletions
@@ -0,0 +1,238 @@
+# 0047: Every module proves itself inside Ekklesia, with one harness the platform publishes
+
+- Status: Proposed
+- Date: 2026-10-05
+- Supersedes: none
+- Related: [ADR 0016](0016-module-database-access.md),
+  [ADR 0021](0021-modules-as-hostable-products.md),
+  [ADR 0028](0028-module-operational-lifecycle.md),
+  [ADR 0030](0030-host-provided-module-object-storage.md),
+  [ADR 0033](0033-display-kiosk-host-contract.md),
+  [ADR 0036](0036-managed-device-identity-and-pairing.md),
+  [ADR 0041](0041-module-browser-contributions.md),
+  [ADR 0043](0043-display-offline-code-fallback.md),
+  [ADR 0044](0044-self-hosted-reference-deployment.md),
+  [ADR 0045](0045-split-dashboard-and-api-origins.md)
+
+## Context
+
+Modules are independently versioned products (ADR 0021). They live in their own repositories, and
+none of them can be built, run, or rendered without Ekklesia. The first production module, Signage,
+reached a church with three defects that only show when it runs **inside** Ekklesia, in a real
+browser:
+
+- every heartbeat its display sent was refused with a 400, because the display and the server
+  disagreed about `null`;
+- a display that failed one request at start-up never recovered without a reload;
+- its media cache leaked one blob per slide per refresh, until the browser refused the display's own
+  requests.
+
+Signage had unit tests, browser tests against a stubbed host, and no CI. Every test passed, because
+each half was tested against its own idea of the other, and nothing ever ran the module in the
+platform it ships into.
+
+The platform's own deployment smoke (ADR 0044, ADR 0045) does run a module in a real browser, but
+only the repository's template, and only far enough to mount it. The instance repository runs the
+smoke against its real modules, but only at release time, after a module has already been tagged.
+Each module repository that wants more than that has to assemble the platform, build an image,
+start a deployment, and drive a browser itself. Signage is about to do exactly that by hand. The
+second module, Classroom, would copy it, and the copies would drift from the platform they test.
+
+## Decision
+
+### The platform publishes a versioned module harness
+
+Each platform release publishes one OCI image, `ekklesia-module-harness:<platform version>`. It is
+built from the platform at that release tag by the platform's own release CI, and pushed to the
+organization's registry. A version is never overwritten. Modules reference it **by digest**.
+
+The harness contains the platform source at that version: its `deploy/` directory, its Dockerfile,
+its lockfile, and its smoke scripts. It also contains the pinned Node.js and pnpm, the browser
+driver, and the pinned headless Chromium image reference. Given a module checkout, it:
+
+1. **assembles** the module into the platform at `modules/<id>`, exactly as an instance does (ADR
+   0044), and installs with `--frozen-lockfile`;
+2. runs the platform's own gates against that set: `pnpm boundaries`, `modules:validate`,
+   `migrations:validate`, and the build;
+3. **builds** the deployment image from the assembled workspace with the platform's Dockerfile;
+4. starts the **reference deployment** in the split topology (ADR 0045), with a disposable
+   PostgreSQL and object store, Caddy's internal CA, and freshly generated secrets;
+5. plans, then **explicitly applies**, migrations against that fresh database. This is the only
+   database the harness ever migrates;
+6. creates the initial owner, grants the module's declared permissions to a test person, and
+   **mounts every UI surface the manifest declares**, in headless Chromium. That covers:
+   - the primary UI in the Dashboard;
+   - the display, paired as a device with no human session (ADR 0036);
+   - each browser contribution, whenever the module that hosts it is present (ADR 0041);
+7. runs a short **display soak**, and the **no-dead-ends** injections below;
+8. tears everything down: containers, networks, volumes, and the work directory.
+
+It **fails** on any of the following:
+
+- a `console.error` or uncaught exception on any page;
+- an unhandled promise rejection;
+- any 4xx or 5xx response that the harness did not itself expect. Expected ones are listed in the
+  harness, for example a display's descriptor before it is paired;
+- a page that never mounts;
+- a failed or refused request (CORS, blocked, network error);
+- a soak leak.
+
+Each failure line names the surface, the request path (never its query), and the step.
+
+### The display soak and the no-dead-ends checks are generic
+
+These checks come from what Signage got wrong, and they apply to any module with a display.
+
+- **Soak**, two minutes by default. The display stays mounted with no remount and no offline
+  indicator. Its realtime streams stay healthy. Live object URLs stay bounded. JavaScript heap
+  growth, measured over DevTools, stays within a fixed ratio.
+- **No dead ends.** One at a time, the harness refuses the first request to each module route the
+  display calls at start-up, and the first realtime connection. Within a bounded time and with no
+  reload, the display must make that request again and succeed. A display has nobody to press
+  Retry.
+
+Module-specific behaviour, such as Signage's heartbeat being recorded, is not guessed by the
+harness. A module declares it (below).
+
+### A module is tested against both ends of its compatibility range
+
+A module's CI runs the harness twice: against the **lowest** and the **newest** published platform
+version within its manifest's `compatibility.platform` range. Testing every version in between adds
+cost and finds little. Testing only the newest misses the floor the manifest promises to work on.
+
+The two versions, and their harness digests, are pinned in the module's repository. The harness
+refuses to run if:
+
+- a pinned version lies outside the module's range;
+- the lowest pinned version is not the range's floor.
+
+Moving to a newer platform is a deliberate, reviewed pin change, as an instance's is (ADR 0044). It
+is never a moving tag.
+
+A frozen install of the module assembled into each of those two platforms needs a lockfile per
+pinned version. The module keeps them (`ci/lock/<platform version>.yaml`), and the harness provides
+the one command that regenerates them inside its own pinned Node.js. A stale lockfile fails the
+frozen install, which is the signal to regenerate it in a reviewed change.
+
+### Modules declare extra smoke steps, beside their manifest rather than in it
+
+A module may add steps of its own in `ekklesia.harness.json`, next to `ekklesia.module.json`. Steps
+are declarative first:
+
+- open a surface;
+- wait for a test id;
+- call one of the module's routes as a named role, expecting a status;
+- soak for N minutes;
+- inject a failure into a named route;
+- check a value in the module's own table, through its own route.
+
+For what a declaration cannot express, a step may name a driver script in the module's repository.
+The harness runs that script with the same DevTools client and the same failure rules.
+
+**This is deliberately not a manifest field.** Manifest versions are strict, hard cutovers that every
+host validates (ADR 0033). A test step is not a runtime contract, and a host must never read one.
+Putting steps in the manifest would force a manifest version bump on every module for something
+that has no meaning outside CI.
+
+### Later: the harness checks what a module actually touches against what it declares
+
+In a later phase, the harness records what the module does while it runs:
+
+- the tables and schemas it reads and writes;
+- the buckets it uses;
+- its outbound network connections, with the egress network blocked and every attempt logged.
+
+It compares them with what the manifest declares (ADR 0016, ADR 0030). A module that touches what it
+did not declare fails. This is the module-trust check. The egress half needs a manifest declaration
+of outbound destinations that does not exist yet, and is a separate decision. This ADR fixes only
+the harness's shape, so that the recording phases can be added without changing how modules call
+it.
+
+### The template's workflow stays short, and calls the harness by digest
+
+`templates/module` ships a workflow that:
+
+- checks out the module with `persist-credentials: false` and pinned actions;
+- runs the module's own unit tests, typecheck, and lint;
+- runs the harness, once per pinned platform version, with one `docker run` of the harness image
+  **by digest**. The module directory is mounted read-only, and the runner's Docker is used exactly
+  as the platform's smoke uses it.
+
+It references no secret, needs no registry credential (the harness image is public within the
+organization), and uses no `pull_request_target`. The trusted-code-only rule for self-hosted runners
+applies to module repositories exactly as it does to the platform (see `docs/development/security.md`).
+
+**Why `docker run`, and not a reusable workflow or a composite action.** The inventoried server is
+Gitea 1.24.2.
+
+- Its runner does support calling a reusable workflow from another repository, by cloning that
+  repository. That support is thinly documented, and has known problems in practice.
+- Actions and reusable workflows from **private** repositories only arrive with Gitea 1.26
+  (go-gitea/gitea#32562).
+- Composite actions from another repository work through absolute `uses:` URLs. That would still
+  wrap the same `docker run`, adding a second pinned reference and nothing else.
+
+An image digest is the one reference that is equally immutable, equally reviewable, and works
+unchanged on Gitea 1.24, Gitea 1.26, and GitHub Actions. A composite action can be added later as a
+convenience over the same image, without changing what a module pins.
+
+### Publishing follows the instance's release pattern
+
+The harness image is pushed only by a `publish` job that runs on `v*` tags, after the platform's CI
+has built and tested that exact image (checked by image ID). That job:
+
+- uses a write-only package token, the only secret in the platform's workflow;
+- refuses to overwrite an existing version;
+- prints the digest.
+
+Pull requests and branch builds build the harness and run it against the template, but never push.
+The registry is reached exactly as the instance's publish job reaches it.
+
+### Adoption
+
+- **Signage** uses an interim CI job until the first harness is published. That job assembles
+  Signage into a pinned platform commit and runs the platform's display-device smoke scenarios.
+  Signage switches to the harness as soon as one is published.
+- **Classroom** follows.
+- **The template** carries the workflow from the first harness release on, so every new module has
+  it from its first commit.
+
+## Consequences
+
+- A module cannot reach a release without having run inside Ekklesia, in a real browser, at both
+  ends of the platform range it claims.
+- The defects that reached the first production Signage deployment fail a module's CI instead:
+  refused requests, dead ends, leaks, and console errors.
+- The platform owns one harness, versioned with itself. Modules do not copy deployment scripts that
+  then drift.
+- Every module's CI needs Docker on its runner, and takes minutes rather than seconds. The harness
+  runs as separate jobs from the unit tests, so a quick failure still reports quickly.
+- The platform's release process gains one published artifact, one credential, and one more failure
+  mode at release time.
+- A module now carries a lockfile for each pinned platform version. The harness makes regenerating
+  them one command, and a stale one fails visibly.
+
+## Alternatives
+
+- **Each module keeps its own copy of the smoke scripts.** Rejected. That is today's situation in
+  embryo, and it drifts.
+- **A reusable workflow in the platform repository.** Rejected for now, for the Gitea 1.24 reasons
+  above. It would also give the harness's behaviour to whichever runner and server version happens
+  to expand it.
+- **Publish the harness as an npm package.** Rejected. The harness needs Docker, a browser, and the
+  platform's deployment files anyway, so a package would ship a fraction of it and assume the rest.
+- **Test only the newest platform version.** Rejected. A module's range floor would go untested
+  until an installation that has not upgraded meets it.
+- **Put smoke steps in the manifest.** Rejected. Manifest versions are hard cutovers with runtime
+  meaning, and test steps have neither.
+- **Run the harness in the platform's CI against every known module.** Rejected. The platform would
+  need every module's source and would fail on modules it does not own. The direction is module
+  tests platform, not the reverse.
+
+## Deferred
+
+- The data-access and egress recording phases, and the manifest declaration egress needs.
+- A composite action over the harness image.
+- Running the harness against a module's declared dependencies at their own pinned versions, for
+  contributions hosted by another module.
+- Visual regression of mounted surfaces.
@@ -55,3 +55,4 @@ ADRs capture durable choices that affect more than one package or phase.
 - [0044: Ekklesia ships one self-hosted reference deployment, composed from pinned sources](0044-self-hosted-reference-deployment.md) — Accepted
 - [0045: The Dashboard and the API may live on separate origins](0045-split-dashboard-and-api-origins.md) — Accepted
 - [0046: Ekklesia's branding is never forced on an installation](0046-installation-owned-branding.md) — Proposed
+- [0047: Every module proves itself inside Ekklesia, with one harness the platform publishes](0047-module-ci-harness.md) — Proposed