2026-10-06 02:00:39 +00:00
2 changed files with 256 additions and 0 deletions
@@ -0,0 +1,255 @@
 # 0047: Every module proves itself inside Ekklesia, with one harness the platform publishes
 - Status: Proposed
 - Date: 2026-10-05
 - Supersedes: none
 - Related: [ADR 0016](0016-module-database-access.md),
  [ADR 0021](0021-modules-as-hostable-products.md),
  [ADR 0028](0028-module-operational-lifecycle.md),
  [ADR 0030](0030-host-provided-module-object-storage.md),
  [ADR 0033](0033-display-kiosk-host-contract.md),
  [ADR 0036](0036-managed-device-identity-and-pairing.md),
  [ADR 0041](0041-module-browser-contributions.md),
  [ADR 0043](0043-display-offline-code-fallback.md),
  [ADR 0044](0044-self-hosted-reference-deployment.md),
  [ADR 0045](0045-split-dashboard-and-api-origins.md)
 ## Context
 Modules are independently versioned products (ADR 0021). They live in their own repositories, and
 none of them can be built, run, or rendered without Ekklesia. The first production module, Signage,
 reached a church with three defects that only show when it runs **inside** Ekklesia, in a real
 browser:
 - every heartbeat its display sent was refused with a 400, because the display and the server
  disagreed about `null`;
 - a display that failed one request at start-up never recovered without a reload;
 - its media cache leaked one blob per slide per refresh, until the browser refused the display's own
  requests.
 Signage had unit tests, browser tests against a stubbed host, and no CI. Every test passed, because
 each half was tested against its own idea of the other, and nothing ever ran the module in the
 platform it ships into.
 The platform's own deployment smoke (ADR 0044, ADR 0045) does run a module in a real browser, but
 only the repository's template, and only far enough to mount it. The instance repository runs the
 smoke against its real modules, but only at release time, after a module has already been tagged.
 Each module repository that wants more than that has to assemble the platform, build an image,
 start a deployment, and drive a browser itself. Signage is about to do exactly that by hand. The
 second module, Classroom, would copy it, and the copies would drift from the platform they test.
 ## Decision
 ### The platform publishes a versioned module harness
 Each platform release publishes one OCI image, `ekklesia-module-harness:<platform version>`. It is
 built from the platform at that release tag by the platform's own release CI, and pushed to the
 organization's registry. A version is never overwritten. Modules reference it **by digest**.
 The harness contains the platform source at that version: its `deploy/` directory, its Dockerfile,
 its lockfile, and its smoke scripts. It also contains the pinned Node.js and pnpm, the browser
 driver, and the pinned headless Chromium image reference. Given a module checkout, it:
 1. **assembles** the module into the platform at `modules/<id>`, exactly as an instance does (ADR
   0044), and installs with `--frozen-lockfile`;
 2. runs the platform's own gates against that set: `pnpm boundaries`, `modules:validate`,
   `migrations:validate`, and the build;
 3. **builds** the deployment image from the assembled workspace with the platform's Dockerfile;
 4. starts the **reference deployment** in the split topology (ADR 0045), with a disposable
   PostgreSQL and object store, Caddy's internal CA, and freshly generated secrets;
 5. plans, then **explicitly applies**, migrations against that fresh database. This is the only
   database the harness ever migrates;
 6. creates the initial owner, grants the module's declared permissions to a test person, and
   **mounts every UI surface the manifest declares**, in headless Chromium. That covers:
   - the primary UI in the Dashboard;
   - the display, paired as a device with no human session (ADR 0036);
   - each browser contribution, whenever the module that hosts it is present (ADR 0041);
 7. runs a short **display soak**, and the **no-dead-ends** injections below;
 8. tears everything down: containers, networks, volumes, and the work directory.
 It **fails** on any of the following:
 - a `console.error` or uncaught exception on any page;
 - an unhandled promise rejection;
 - any 4xx or 5xx response that the harness did not itself expect. Expected ones are listed in the
  harness, for example a display's descriptor before it is paired;
 - a page that never mounts;
 - a failed or refused request (CORS, blocked, network error);
 - a soak leak.
 Each failure line names the surface, the request path (never its query), and the step.
 ### The display soak and the no-dead-ends checks are generic
 These checks come from what Signage got wrong, and they apply to any module with a display.
 - **Soak**, two minutes by default. The display stays mounted with no remount and no offline
  indicator. Its realtime streams stay healthy. Live object URLs stay bounded. JavaScript heap
  growth, measured over DevTools, stays within a fixed ratio.
 - **No dead ends.** One at a time, the harness refuses the first request to each module route the
  display calls at start-up, and the first realtime connection. Within a bounded time and with no
  reload, the display must make that request again and succeed. A display has nobody to press
  Retry.
 Module-specific behaviour, such as Signage's heartbeat being recorded, is not guessed by the
 harness. A module declares it (below).
 ### A module is tested against both ends of its compatibility range
 A module's CI runs the harness twice: against the **lowest** and the **newest** published platform
 version within its manifest's `compatibility.platform` range. Testing every version in between adds
 cost and finds little. Testing only the newest misses the floor the manifest promises to work on.
 The two versions, and their harness digests, are pinned in the module's repository. The harness
 refuses to run if:
 - a pinned version lies outside the module's range;
 - the lowest pinned version is not the range's floor.
 Moving to a newer platform is a deliberate, reviewed pin change, as an instance's is (ADR 0044). It
 is never a moving tag.
 A frozen install of the module assembled into each of those two platforms needs a lockfile per
 pinned version. The module keeps them (`ci/lock/<platform version>.yaml`), and the harness provides
 the one command that regenerates them inside its own pinned Node.js. A stale lockfile fails the
 frozen install, which is the signal to regenerate it in a reviewed change.
 ### Modules declare extra smoke steps, beside their manifest rather than in it
 A module may add steps of its own in `ekklesia.harness.json`, next to `ekklesia.module.json`. Steps
 are declarative first:
 - open a surface;
 - wait for a test id;
 - call one of the module's routes as a named role, expecting a status;
 - soak for N minutes;
 - inject a failure into a named route;
 - check a value in the module's own table, through its own route.
 For what a declaration cannot express, a step may name a driver script in the module's repository.
 The harness runs that script with the same DevTools client and the same failure rules.
 **This is deliberately not a manifest field.** Manifest versions are strict, hard cutovers that every
 host validates (ADR 0033). A test step is not a runtime contract, and a host must never read one.
 Putting steps in the manifest would force a manifest version bump on every module for something
 that has no meaning outside CI.
 ### Later: the harness checks what a module actually touches against what it declares
 In a later phase, the harness records what the module does while it runs:
 - the tables and schemas it reads and writes;
 - the buckets it uses;
 - its outbound network connections, with the egress network blocked and every attempt logged.
 It compares them with what the manifest declares (ADR 0016, ADR 0030). A module that touches what it
 did not declare fails. This is the module-trust check. The egress half needs a manifest declaration
 of outbound destinations that does not exist yet, and is a separate decision. This ADR fixes only
 the harness's shape, so that the recording phases can be added without changing how modules call
 it.
 ### The template's workflow stays short, and calls the harness by digest
 `templates/module` ships a workflow that:
 - checks out the module with `persist-credentials: false` and pinned actions;
 - runs the module's own unit tests, typecheck, and lint;
 - sweeps stale CI stacks from the runner, then runs the harness, once per pinned platform version,
  with one `docker run` of the harness image **by digest**, and finally, always, removes this job's
  own stacks. The module directory is mounted read-only, and the runner's Docker is used exactly
  as the platform's smoke uses it.
 It references no secret, needs no registry credential (the harness image is public within the
 organization), and uses no `pull_request_target`. The trusted-code-only rule for self-hosted runners
 applies to module repositories exactly as it does to the platform (see `docs/development/security.md`).
 **Why `docker run`, and not a reusable workflow or a composite action.** The inventoried server is
 Gitea 1.24.2.
 - Its runner does support calling a reusable workflow from another repository, by cloning that
  repository. That support is thinly documented, and has known problems in practice.
 - Actions and reusable workflows from **private** repositories only arrive with Gitea 1.26
  (go-gitea/gitea#32562).
 - Composite actions from another repository work through absolute `uses:` URLs. That would still
  wrap the same `docker run`, adding a second pinned reference and nothing else.
 An image digest is the one reference that is equally immutable, equally reviewable, and works
 unchanged on Gitea 1.24, Gitea 1.26, and GitHub Actions. A composite action can be added later as a
 convenience over the same image, without changing what a module pins.
 ### Nothing the harness starts outlives the job that started it
 The harness starts Docker stacks on a shared runner, and a runner can kill a job outright: a
 cancelled job, or the runner's own time limit, since Gitea 1.24 ignores `timeout-minutes`. A script's
 own cleanup trap does not run then. The platform's CI leaked a whole migration stack that way, and it
 ran until someone removed it by hand. So the harness follows the platform's rule
 (`scripts/ci-sweep.sh`, `docs/development/security.md`):
 - everything it starts is named and labeled as a CI stack of its run, attempt, and job key;
 - it removes its own stacks in a trap;
 - the template workflow runs the harness's **stale sweep** before the harness starts, and its
  **job sweep** in a final `if: always()` step, so a killed run's stacks are removed by that run's
  last step, or failing that by the next job on the runner.
 The sweep ships inside the harness image, so a module never copies it.
 ### Publishing follows the instance's release pattern
 The harness image is pushed only by a `publish` job that runs on `v*` tags, after the platform's CI
 has built and tested that exact image (checked by image ID). That job:
 - uses a write-only package token, the only secret in the platform's workflow;
 - refuses to overwrite an existing version;
 - prints the digest.
 Pull requests and branch builds build the harness and run it against the template, but never push.
 The registry is reached exactly as the instance's publish job reaches it.
 ### Adoption
 - **Signage** uses an interim CI job until the first harness is published. That job assembles
  Signage into a pinned platform commit and runs the platform's display-device smoke scenarios.
  Signage switches to the harness as soon as one is published.
 - **Classroom** follows.
 - **The template** carries the workflow from the first harness release on, so every new module has
  it from its first commit.
 ## Consequences
 - A module cannot reach a release without having run inside Ekklesia, in a real browser, at both
  ends of the platform range it claims.
 - The defects that reached the first production Signage deployment fail a module's CI instead:
  refused requests, dead ends, leaks, and console errors.
 - The platform owns one harness, versioned with itself. Modules do not copy deployment scripts that
  then drift.
 - Every module's CI needs Docker on its runner, and takes minutes rather than seconds. The harness
  runs as separate jobs from the unit tests, so a quick failure still reports quickly.
 - The platform's release process gains one published artifact, one credential, and one more failure
  mode at release time.
 - A module now carries a lockfile for each pinned platform version. The harness makes regenerating
  them one command, and a stale one fails visibly.
 ## Alternatives
 - **Each module keeps its own copy of the smoke scripts.** Rejected. That is today's situation in
  embryo, and it drifts.
 - **A reusable workflow in the platform repository.** Rejected for now, for the Gitea 1.24 reasons
  above. It would also give the harness's behaviour to whichever runner and server version happens
  to expand it.
 - **Publish the harness as an npm package.** Rejected. The harness needs Docker, a browser, and the
  platform's deployment files anyway, so a package would ship a fraction of it and assume the rest.
 - **Test only the newest platform version.** Rejected. A module's range floor would go untested
  until an installation that has not upgraded meets it.
 - **Put smoke steps in the manifest.** Rejected. Manifest versions are hard cutovers with runtime
  meaning, and test steps have neither.
 - **Run the harness in the platform's CI against every known module.** Rejected. The platform would
  need every module's source and would fail on modules it does not own. The direction is module
  tests platform, not the reverse.
 ## Deferred
 - The data-access and egress recording phases, and the manifest declaration egress needs.
 - A composite action over the harness image.
 - Running the harness against a module's declared dependencies at their own pinned versions, for
  contributions hosted by another module.
 - Visual regression of mounted surfaces.
@@ -55,3 +55,4 @@ ADRs capture durable choices that affect more than one package or phase.
 - [0044: Ekklesia ships one self-hosted reference deployment, composed from pinned sources](0044-self-hosted-reference-deployment.md) — Accepted
 - [0045: The Dashboard and the API may live on separate origins](0045-split-dashboard-and-api-origins.md) — Accepted
 - [0046: Ekklesia's branding is never forced on an installation](0046-installation-owned-branding.md) — Proposed
 - [0047: Every module proves itself inside Ekklesia, with one harness the platform publishes](0047-module-ci-harness.md) — Proposed