ekklesiaandClaude Opus 5.5
3827841205
CI / verify (pull_request) Successful in 53s
CI / infrastructure-smoke (pull_request) Successful in 21s
CI / migration-smoke (pull_request) Successful in 1m49s
CI / image-build (pull_request) Successful in 19s
CI / deploy-smoke (same-origin) (pull_request) Successful in 49s
CI / deploy-smoke (split) (pull_request) Canceled after 18s
Signage reached production with three defects that appear only inside Ekklesia in a real browser, and no CI that ran it there. The ADR proposes: - a versioned harness image, ekklesia-module-harness:<platform version>, built and pushed by the platform's release CI; - what the harness does: assemble a module into that platform, validate and build it, migrate a disposable database, start the split reference deployment, and mount every declared UI surface in headless Chromium; - what fails it: console errors, unexpected 4xx/5xx, unhandled rejections, a display soak, and generic no-dead-ends injections; - testing at the lowest and newest platform versions of a module's range; - extra smoke steps in ekklesia.harness.json, beside the manifest rather than in it; - a later data-access and egress check against declarations; - a short template workflow that calls the harness by digest with one `docker run`. Not a reusable workflow: thinly supported on Gitea 1.24, and private sources only arrive in 1.26. Co-Authored-By: Claude Opus 5.5 <[email protected]>
239 lines
13 KiB
Markdown
239 lines
13 KiB
Markdown
# 0047: Every module proves itself inside Ekklesia, with one harness the platform publishes
|
|
|
|
- Status: Proposed
|
|
- Date: 2026-10-05
|
|
- Supersedes: none
|
|
- Related: [ADR 0016](0016-module-database-access.md),
|
|
[ADR 0021](0021-modules-as-hostable-products.md),
|
|
[ADR 0028](0028-module-operational-lifecycle.md),
|
|
[ADR 0030](0030-host-provided-module-object-storage.md),
|
|
[ADR 0033](0033-display-kiosk-host-contract.md),
|
|
[ADR 0036](0036-managed-device-identity-and-pairing.md),
|
|
[ADR 0041](0041-module-browser-contributions.md),
|
|
[ADR 0043](0043-display-offline-code-fallback.md),
|
|
[ADR 0044](0044-self-hosted-reference-deployment.md),
|
|
[ADR 0045](0045-split-dashboard-and-api-origins.md)
|
|
|
|
## Context
|
|
|
|
Modules are independently versioned products (ADR 0021). They live in their own repositories, and
|
|
none of them can be built, run, or rendered without Ekklesia. The first production module, Signage,
|
|
reached a church with three defects that only show when it runs **inside** Ekklesia, in a real
|
|
browser:
|
|
|
|
- every heartbeat its display sent was refused with a 400, because the display and the server
|
|
disagreed about `null`;
|
|
- a display that failed one request at start-up never recovered without a reload;
|
|
- its media cache leaked one blob per slide per refresh, until the browser refused the display's own
|
|
requests.
|
|
|
|
Signage had unit tests, browser tests against a stubbed host, and no CI. Every test passed, because
|
|
each half was tested against its own idea of the other, and nothing ever ran the module in the
|
|
platform it ships into.
|
|
|
|
The platform's own deployment smoke (ADR 0044, ADR 0045) does run a module in a real browser, but
|
|
only the repository's template, and only far enough to mount it. The instance repository runs the
|
|
smoke against its real modules, but only at release time, after a module has already been tagged.
|
|
Each module repository that wants more than that has to assemble the platform, build an image,
|
|
start a deployment, and drive a browser itself. Signage is about to do exactly that by hand. The
|
|
second module, Classroom, would copy it, and the copies would drift from the platform they test.
|
|
|
|
## Decision
|
|
|
|
### The platform publishes a versioned module harness
|
|
|
|
Each platform release publishes one OCI image, `ekklesia-module-harness:<platform version>`. It is
|
|
built from the platform at that release tag by the platform's own release CI, and pushed to the
|
|
organization's registry. A version is never overwritten. Modules reference it **by digest**.
|
|
|
|
The harness contains the platform source at that version: its `deploy/` directory, its Dockerfile,
|
|
its lockfile, and its smoke scripts. It also contains the pinned Node.js and pnpm, the browser
|
|
driver, and the pinned headless Chromium image reference. Given a module checkout, it:
|
|
|
|
1. **assembles** the module into the platform at `modules/<id>`, exactly as an instance does (ADR
|
|
0044), and installs with `--frozen-lockfile`;
|
|
2. runs the platform's own gates against that set: `pnpm boundaries`, `modules:validate`,
|
|
`migrations:validate`, and the build;
|
|
3. **builds** the deployment image from the assembled workspace with the platform's Dockerfile;
|
|
4. starts the **reference deployment** in the split topology (ADR 0045), with a disposable
|
|
PostgreSQL and object store, Caddy's internal CA, and freshly generated secrets;
|
|
5. plans, then **explicitly applies**, migrations against that fresh database. This is the only
|
|
database the harness ever migrates;
|
|
6. creates the initial owner, grants the module's declared permissions to a test person, and
|
|
**mounts every UI surface the manifest declares**, in headless Chromium. That covers:
|
|
- the primary UI in the Dashboard;
|
|
- the display, paired as a device with no human session (ADR 0036);
|
|
- each browser contribution, whenever the module that hosts it is present (ADR 0041);
|
|
7. runs a short **display soak**, and the **no-dead-ends** injections below;
|
|
8. tears everything down: containers, networks, volumes, and the work directory.
|
|
|
|
It **fails** on any of the following:
|
|
|
|
- a `console.error` or uncaught exception on any page;
|
|
- an unhandled promise rejection;
|
|
- any 4xx or 5xx response that the harness did not itself expect. Expected ones are listed in the
|
|
harness, for example a display's descriptor before it is paired;
|
|
- a page that never mounts;
|
|
- a failed or refused request (CORS, blocked, network error);
|
|
- a soak leak.
|
|
|
|
Each failure line names the surface, the request path (never its query), and the step.
|
|
|
|
### The display soak and the no-dead-ends checks are generic
|
|
|
|
These checks come from what Signage got wrong, and they apply to any module with a display.
|
|
|
|
- **Soak**, two minutes by default. The display stays mounted with no remount and no offline
|
|
indicator. Its realtime streams stay healthy. Live object URLs stay bounded. JavaScript heap
|
|
growth, measured over DevTools, stays within a fixed ratio.
|
|
- **No dead ends.** One at a time, the harness refuses the first request to each module route the
|
|
display calls at start-up, and the first realtime connection. Within a bounded time and with no
|
|
reload, the display must make that request again and succeed. A display has nobody to press
|
|
Retry.
|
|
|
|
Module-specific behaviour, such as Signage's heartbeat being recorded, is not guessed by the
|
|
harness. A module declares it (below).
|
|
|
|
### A module is tested against both ends of its compatibility range
|
|
|
|
A module's CI runs the harness twice: against the **lowest** and the **newest** published platform
|
|
version within its manifest's `compatibility.platform` range. Testing every version in between adds
|
|
cost and finds little. Testing only the newest misses the floor the manifest promises to work on.
|
|
|
|
The two versions, and their harness digests, are pinned in the module's repository. The harness
|
|
refuses to run if:
|
|
|
|
- a pinned version lies outside the module's range;
|
|
- the lowest pinned version is not the range's floor.
|
|
|
|
Moving to a newer platform is a deliberate, reviewed pin change, as an instance's is (ADR 0044). It
|
|
is never a moving tag.
|
|
|
|
A frozen install of the module assembled into each of those two platforms needs a lockfile per
|
|
pinned version. The module keeps them (`ci/lock/<platform version>.yaml`), and the harness provides
|
|
the one command that regenerates them inside its own pinned Node.js. A stale lockfile fails the
|
|
frozen install, which is the signal to regenerate it in a reviewed change.
|
|
|
|
### Modules declare extra smoke steps, beside their manifest rather than in it
|
|
|
|
A module may add steps of its own in `ekklesia.harness.json`, next to `ekklesia.module.json`. Steps
|
|
are declarative first:
|
|
|
|
- open a surface;
|
|
- wait for a test id;
|
|
- call one of the module's routes as a named role, expecting a status;
|
|
- soak for N minutes;
|
|
- inject a failure into a named route;
|
|
- check a value in the module's own table, through its own route.
|
|
|
|
For what a declaration cannot express, a step may name a driver script in the module's repository.
|
|
The harness runs that script with the same DevTools client and the same failure rules.
|
|
|
|
**This is deliberately not a manifest field.** Manifest versions are strict, hard cutovers that every
|
|
host validates (ADR 0033). A test step is not a runtime contract, and a host must never read one.
|
|
Putting steps in the manifest would force a manifest version bump on every module for something
|
|
that has no meaning outside CI.
|
|
|
|
### Later: the harness checks what a module actually touches against what it declares
|
|
|
|
In a later phase, the harness records what the module does while it runs:
|
|
|
|
- the tables and schemas it reads and writes;
|
|
- the buckets it uses;
|
|
- its outbound network connections, with the egress network blocked and every attempt logged.
|
|
|
|
It compares them with what the manifest declares (ADR 0016, ADR 0030). A module that touches what it
|
|
did not declare fails. This is the module-trust check. The egress half needs a manifest declaration
|
|
of outbound destinations that does not exist yet, and is a separate decision. This ADR fixes only
|
|
the harness's shape, so that the recording phases can be added without changing how modules call
|
|
it.
|
|
|
|
### The template's workflow stays short, and calls the harness by digest
|
|
|
|
`templates/module` ships a workflow that:
|
|
|
|
- checks out the module with `persist-credentials: false` and pinned actions;
|
|
- runs the module's own unit tests, typecheck, and lint;
|
|
- runs the harness, once per pinned platform version, with one `docker run` of the harness image
|
|
**by digest**. The module directory is mounted read-only, and the runner's Docker is used exactly
|
|
as the platform's smoke uses it.
|
|
|
|
It references no secret, needs no registry credential (the harness image is public within the
|
|
organization), and uses no `pull_request_target`. The trusted-code-only rule for self-hosted runners
|
|
applies to module repositories exactly as it does to the platform (see `docs/development/security.md`).
|
|
|
|
**Why `docker run`, and not a reusable workflow or a composite action.** The inventoried server is
|
|
Gitea 1.24.2.
|
|
|
|
- Its runner does support calling a reusable workflow from another repository, by cloning that
|
|
repository. That support is thinly documented, and has known problems in practice.
|
|
- Actions and reusable workflows from **private** repositories only arrive with Gitea 1.26
|
|
(go-gitea/gitea#32562).
|
|
- Composite actions from another repository work through absolute `uses:` URLs. That would still
|
|
wrap the same `docker run`, adding a second pinned reference and nothing else.
|
|
|
|
An image digest is the one reference that is equally immutable, equally reviewable, and works
|
|
unchanged on Gitea 1.24, Gitea 1.26, and GitHub Actions. A composite action can be added later as a
|
|
convenience over the same image, without changing what a module pins.
|
|
|
|
### Publishing follows the instance's release pattern
|
|
|
|
The harness image is pushed only by a `publish` job that runs on `v*` tags, after the platform's CI
|
|
has built and tested that exact image (checked by image ID). That job:
|
|
|
|
- uses a write-only package token, the only secret in the platform's workflow;
|
|
- refuses to overwrite an existing version;
|
|
- prints the digest.
|
|
|
|
Pull requests and branch builds build the harness and run it against the template, but never push.
|
|
The registry is reached exactly as the instance's publish job reaches it.
|
|
|
|
### Adoption
|
|
|
|
- **Signage** uses an interim CI job until the first harness is published. That job assembles
|
|
Signage into a pinned platform commit and runs the platform's display-device smoke scenarios.
|
|
Signage switches to the harness as soon as one is published.
|
|
- **Classroom** follows.
|
|
- **The template** carries the workflow from the first harness release on, so every new module has
|
|
it from its first commit.
|
|
|
|
## Consequences
|
|
|
|
- A module cannot reach a release without having run inside Ekklesia, in a real browser, at both
|
|
ends of the platform range it claims.
|
|
- The defects that reached the first production Signage deployment fail a module's CI instead:
|
|
refused requests, dead ends, leaks, and console errors.
|
|
- The platform owns one harness, versioned with itself. Modules do not copy deployment scripts that
|
|
then drift.
|
|
- Every module's CI needs Docker on its runner, and takes minutes rather than seconds. The harness
|
|
runs as separate jobs from the unit tests, so a quick failure still reports quickly.
|
|
- The platform's release process gains one published artifact, one credential, and one more failure
|
|
mode at release time.
|
|
- A module now carries a lockfile for each pinned platform version. The harness makes regenerating
|
|
them one command, and a stale one fails visibly.
|
|
|
|
## Alternatives
|
|
|
|
- **Each module keeps its own copy of the smoke scripts.** Rejected. That is today's situation in
|
|
embryo, and it drifts.
|
|
- **A reusable workflow in the platform repository.** Rejected for now, for the Gitea 1.24 reasons
|
|
above. It would also give the harness's behaviour to whichever runner and server version happens
|
|
to expand it.
|
|
- **Publish the harness as an npm package.** Rejected. The harness needs Docker, a browser, and the
|
|
platform's deployment files anyway, so a package would ship a fraction of it and assume the rest.
|
|
- **Test only the newest platform version.** Rejected. A module's range floor would go untested
|
|
until an installation that has not upgraded meets it.
|
|
- **Put smoke steps in the manifest.** Rejected. Manifest versions are hard cutovers with runtime
|
|
meaning, and test steps have neither.
|
|
- **Run the harness in the platform's CI against every known module.** Rejected. The platform would
|
|
need every module's source and would fail on modules it does not own. The direction is module
|
|
tests platform, not the reverse.
|
|
|
|
## Deferred
|
|
|
|
- The data-access and egress recording phases, and the manifest declaration egress needs.
|
|
- A composite action over the harness image.
|
|
- Running the harness against a module's declared dependencies at their own pinned versions, for
|
|
contributions hosted by another module.
|
|
- Visual regression of mounted surfaces.
|