ekklesiaandClaude Opus 5.5
317a4ce410
CI / verify (pull_request) Successful in 1m4s
CI / infrastructure-smoke (pull_request) Successful in 38s
CI / migration-smoke (pull_request) Successful in 1m54s
CI / image-build (pull_request) Successful in 22s
CI / deploy-smoke (same-origin) (pull_request) Successful in 51s
CI / deploy-smoke (split) (pull_request) Successful in 1m37s
Narrow contract and implementation adjustments from review. No SDK, manifest, packaging, or Signage work. Callback-error precedence: - A transaction or savepoint callback that rejects keeps its own error, even while sibling work it started is still settling (one failed half of a Promise.all). Core drains that work, rolls back, and never commits or releases. - A callback that resolves with work outstanding (a forgotten await) still fails with MODULE_DATABASE_SCOPE_WORK_OUTSTANDING and rolls back. Transaction-control SQL: - ADR 0050 forbids it through query() as a conformance rule for trusted code, not a security guarantee. Core does not parse SQL, so PostgreSQL has run the statement before Core sees it. Core detects the violation from the command tag or the transaction status, and claims nothing it cannot do: a module's own COMMIT has committed. - After a violation, the transaction's scopes refuse every later statement without sending it (it would otherwise autocommit). Core never commits the transaction itself, and the connection is destroyed rather than reused, for top-level violations too. - The commit and rollback guarantees apply to conforming use. Duplicate object-mode columns: documented as found only after execution, so a mutation can have taken effect; atomicity needs an ABI transaction. Tests prove each of these against PostgreSQL: Promise.all failures in a transaction and a savepoint keep 23505; a forgotten await still rolls back; a module COMMIT's prior work is committed while nothing after it runs; a backend PID changes after every violation; duplicate RETURNING columns persist at the top level and roll back in a transaction. ADR 0050 stays Accepted: the executor conforms to the revised text. Co-Authored-By: Claude Opus 5.5 <[email protected]>
661 lines
50 KiB
Markdown
661 lines
50 KiB
Markdown
# Testing
|
||
|
||
Ekklesia separates fast code-level evidence from live infrastructure evidence.
|
||
|
||
## Unit and repository verification
|
||
|
||
```bash
|
||
pnpm test
|
||
pnpm verify
|
||
```
|
||
|
||
These commands exercise package tests, repository boundaries, type checking, offline Core and
|
||
module migration discovery, identity schema drift detection, formatting, linting, and builds.
|
||
Migration-runner unit tests use a narrow database/session port to prove deterministic planning,
|
||
approval, transaction orchestration, cleanup, and sanitized errors. They do not claim to prove
|
||
PostgreSQL behavior. Identity unit tests cover branded opaque-ID contracts, principal shapes,
|
||
configuration validation (secret rules, canonical-origin-only base URL and trusted origins, and the
|
||
full loopback/unspecified classifier production rejects — `.localhost`, all of `127.0.0.0/8`,
|
||
`0.0.0.0`, `::1` compressed and expanded, `::`, and IPv4-mapped forms), bounded/validated
|
||
connection and operation (`query_timeout`/`statement_timeout`) database timeouts, the Core identity
|
||
runtime (explicit construction, exact schema mapping, only email/password and rate limiting
|
||
enabled, an explicit `advanced.disableCSRFCheck`/`disableOriginCheck: false` pair, an explicit
|
||
error-only sanitized logger that strips Better Auth's own message and arguments regardless of what
|
||
they contain, pinned 8–128 password-length bounds, sanitized failures, exact whole-name
|
||
session-cookie detection that never fires on a prefixed, suffixed, or differently-cased lookalike,
|
||
and that a broken session store during principal resolution is never silently reported as anonymous
|
||
while an unrelated cookie performs no identity database work at all), the PostgreSQL transport
|
||
policy (`core/tests/database-transport.test.ts`: mode validation including every loose or aliased
|
||
spelling, production's plaintext refusal with no loopback exception, connection-string
|
||
TLS-parameter rejection, and that no configuration path yields `rejectUnauthorized: false` or leaks
|
||
CA material into an error), and Core migration discovery (deterministic ordering, the reserved
|
||
`core` identity, and collision rejection) — all without a live database. `apps/api`'s own unit tests cover its pure Hono app factory (route shapes, module
|
||
route mounting/enforcement/CORS/response sanitization, CORS allow/deny/preflight, deterministic 404
|
||
and sanitized error responses), its runtime configuration, its idempotent shutdown lifecycle
|
||
(concurrent callers share one server-close-then-module-runtime-stop-then-pool-close promise; a
|
||
module-stop failure never prevents pool cleanup; a startup failure after the pool was already
|
||
created still closes it, stopping the module runtime first if it had already started;
|
||
`SIGINT`/`SIGTERM` wiring is exercised through an injectable exit function rather than the real
|
||
`process.exit`), and that importing any of its modules — including the executable server module
|
||
without calling its exported `main` — never reads the environment, connects to PostgreSQL, imports
|
||
a module entrypoint, or starts a server. Real, working (non-throwing) module fixtures prove
|
||
production module-runtime startup/shutdown ordering and cleanup against a mocked pool/Better
|
||
Auth/server — see [Authorization](authorization.md#tests).
|
||
|
||
Phase 7B pass 1 adds browser-authentication evidence
|
||
([ADR 0034](../architecture/decisions/0034-host-owned-browser-authentication.md)). API tests pin the
|
||
two exact browser-boundary exceptions. Any body-bearing successful `2xx` from
|
||
`POST /api/auth/sign-in/email` preserves Better Auth's `Set-Cookie` and status but returns fixed
|
||
`{"ok":true}`, with no token/user body. `/api/auth/get-session`, `/api/auth/list-sessions`,
|
||
`/api/auth/update-session`, and `/api/auth/change-password` each receive the identical sanitized
|
||
`404` without invoking the identity handler; failures and every other supported auth route retain
|
||
their native semantics. The live browser transport additionally proves the host-only cookie and
|
||
that no token, session, user, or email appears in the sign-in body.
|
||
|
||
Shell protocol and React tests cover principal-first anonymous/authenticated/failure states; the
|
||
labelled email/password form and keyboard submission; autocomplete, pending state, and same-tick
|
||
duplicate suppression; password clearing; fixed credential-versus-service errors; no credential or
|
||
raw exception logging; fresh principal and navigation resolution after login; sign-out followed by
|
||
principal resolution; and retry without a full-page reload.
|
||
|
||
The same shell suites exercise session transitions under repeated timing: active module cleanup and
|
||
realtime teardown on sign-out/verified expiry; stale attempts after unmount; platform bootstrap
|
||
`401` mapping directly to unauthenticated; and deduplicated principal revalidation for both module
|
||
`401` and `403`. They prove an authenticated recheck preserves the module's original response,
|
||
session, and mounted UI; an anonymous recheck unmounts once and shows login; a revalidation failure
|
||
is a sanitized host failure; and a response from an already-unmounted frame cannot overwrite a newer
|
||
transition. Return/callback input is not part of the browser contract, and tests pin that external,
|
||
protocol-relative, backslash-prefixed, and scheme-bearing values cannot become navigation.
|
||
Modularity checks keep authentication out of the module UI SDK/context and Dashboard login code out
|
||
of the separate display bundle.
|
||
|
||
Phase 8 adds evidence for the five platform prerequisites.
|
||
|
||
**Design tokens and shared elements** (ADR 0039). The token registry's tests prove it is
|
||
deterministic and enumerable, that every token derives a unique `--ekklesia-*` identifier from its
|
||
name and carries a label, description, and category, that each status family pairs a colour with a
|
||
non-colour symbol, and that a duplicate name, an unknown category, an invalid or single-segment name,
|
||
and a value carrying a declaration terminator, a rule block, a comment, an at-rule, `url(`,
|
||
`!important`, an escape, a control character, or markup are each rejected — with an error that never
|
||
echoes the rejected declaration. The elements' tests assert accessible structure (label/hint/error
|
||
association, icon-button naming, tab roles and arrow-key movement, announced error and loading
|
||
states), that status never depends on colour alone, that a dialog renders in place rather than
|
||
portalling out of an isolated root, and — mechanically — that the stylesheet and every component
|
||
source contain no literal colour, font stack, or theme value and reference only registered tokens.
|
||
Both packages compile module-shaped source under the exact browser project a module uses, where
|
||
`process.env` must fail to typecheck, so the browser boundary is measured rather than asserted.
|
||
|
||
**Module stylesheets and artifact isolation** (ADR 0040). The builder's tests prove a plain
|
||
stylesheet is compiled into the declared artifact with no `.css` emitted beside it, that a local
|
||
`@import` is inlined, and that an unresolved `@import`, a stylesheet declaring an `--ekklesia-*`
|
||
variable, and a non-CSS dialect are each rejected with their own sanitized code — while a private
|
||
alias derived from a registered token is accepted. The loader's tests prove every artifact mounts
|
||
inside an open shadow root, that the artifact receives that root rather than the host's element, that
|
||
exactly one `<style>` is installed per mount and never accumulates across remounts, that the host's
|
||
base rules precede the artifact's so a display can override `:host`, that unmount removes the whole
|
||
root even when the module's cleanup throws, that a failed mount leaves neither DOM nor styles behind,
|
||
that an environment without Shadow DOM fails closed rather than mounting unisolated, and that an
|
||
artifact built against the previous ABI is refused with `MODULE_UI_ABI_UNSUPPORTED`. The template's
|
||
suite builds real React artifacts through the canonical pipeline and asserts the compiled stylesheet
|
||
travels inside the one declared file.
|
||
|
||
**Browser contributions** (ADR 0041). Manifest tests cover namespacing, the single local segment,
|
||
duplicate ids and artifact paths, collision with an entrypoint, and a contribution targeting a
|
||
capability its own module provides. Discovery tests prove a contribution artifact inherits the exact
|
||
path, symlink, containment, extension, and file-kind protections every other compiled entrypoint has,
|
||
including two contributions linked to one file. Publication tests prove a contribution is published
|
||
only when another eligible module provides its capability *within the declared range*, that the
|
||
contributing module never sees its own contribution, that a lookup fails closed on an unknown,
|
||
mismatched, or non-string identity, and that the published record carries no domain vocabulary at
|
||
all. Route tests prove the artifact is served byte-for-byte at a URL that cannot select a neighbour,
|
||
and that an unknown id, another module's id, a traversal, a source map, a wrong version, an absent
|
||
provider, and an incompatible provider all collapse into one indistinguishable `404`. The loader's
|
||
tests cover mount, update-without-remount, a renderer that declares no `update`, deterministic and
|
||
idempotent unmount, a sanitized update failure, and input that is not genuinely JSON — a `Date`, a
|
||
function, `NaN`, `undefined`, and a cycle — refused *before* anything is imported and without ever
|
||
echoing the consumer's data. The flagship shell fixture drives the whole relationship: a `signage`
|
||
module discovers, mounts, updates, and unmounts an `announcements` React renderer with neither module
|
||
importing the other, the consumer seeing identity but never a URL, the host resolving publication,
|
||
and a contribution id the server withheld refused without a fetch.
|
||
|
||
**Bounded streaming uploads** (ADR 0042). Contract tests pin the two ceilings, that `maxBodyBytes` is
|
||
checked against the ceiling the declared mode selects, that 32 MiB holds a 25 MiB image and one byte
|
||
more is refused, and that no mode admits an unbounded body. HTTP tests cover exact-boundary success,
|
||
a declared `Content-Length` refused before the handler runs, a misleading smaller `Content-Length`
|
||
that cannot smuggle a megabyte past the limit, a chunked overflow, a `413` that still wins when the
|
||
handler swallowed the stream error, proof that the body is genuinely not buffered (a handler sees the
|
||
first chunk before the last is produced), an aborted upload surfacing as an ordinary failure rather
|
||
than a `413`, an unread body being released, authorization ordering in both directions, and a
|
||
rejection that names no limit, size, or body content. The live suite uploads a real 25 MiB body over
|
||
a real socket end to end and proves the server is healthy immediately afterwards.
|
||
|
||
**Display offline fallback** (ADR 0043). The worker's caching logic is exercised directly, because it
|
||
touches no global: first online load, offline reload after one successful visit, a missing uncached
|
||
artifact failing honestly, contributed-renderer fallback, version upgrade with pruning that keeps
|
||
every artifact of the current version and never touches another module's, superseded cache
|
||
generations removed while unrelated ones survive, network-first behaviour picking up rebuilt bytes,
|
||
and an explicit inventory proving the descriptor, principal, module API, realtime stream, pairing
|
||
exchange, auth routes, cross-origin signed object URLs, and the worker itself are **never** stored —
|
||
so a revoked device's refusal is what a reconnected display sees. Page-level tests cover the quiet
|
||
offline indicator, a cache fallback reported while the browser still claims to be online, automatic
|
||
re-resolution when the network returns, and a revoked device dropping back to pairing on reconnect.
|
||
|
||
`pnpm verify` is intentionally Docker-free. It never plans or applies migrations against a live
|
||
database.
|
||
|
||
## Live PostgreSQL migration evidence
|
||
|
||
Docker and the Docker Compose plugin are required for the live suite:
|
||
|
||
```bash
|
||
pnpm test:migrations:integration
|
||
```
|
||
|
||
The command first builds the workspace through the repository's Turbo build graph, then starts the
|
||
repository's pinned PostgreSQL image in a unique Compose project, waits for bounded health
|
||
readiness, runs the production PostgreSQL adapter and migration services, and removes the
|
||
container, network, and volume. A build failure occurs before Docker provisioning. Each scenario
|
||
also receives a safely named disposable database. Direct host runs use a Docker-assigned random
|
||
port bound only to `127.0.0.1`; a containerized Gitea job joins the isolated Compose network and
|
||
uses the PostgreSQL service name.
|
||
|
||
In CI, a stack whose job was killed before its own cleanup ran is still removed: by that job's final
|
||
`if: always()` step, or by the stale sweep before a later job starts (`scripts/ci-sweep.sh`, see
|
||
[Disposable stacks never outlive their job](security.md#disposable-stacks-never-outlive-their-job)).
|
||
`bash scripts/test-ci-sweep.sh` proves what the sweep removes and what it leaves alone, against real
|
||
Docker resources.
|
||
|
||
The live suite proves fresh read-only planning, initial and incremental application, genuine
|
||
no-op behavior, exact applied checksums and history, edited or corrupt history rejection,
|
||
whole-batch DDL rollback, Drizzle breakpoint execution, real advisory-lock concurrency and
|
||
timeouts, lock release after failure, exact destructive approvals, and public-error sanitization.
|
||
It does not make migrations automatic during Core startup.
|
||
|
||
It also proves ADR 0048's retained history for an uninstalled module end to end, with real
|
||
discovery: Core's committed migrations (the newest one withheld at first) and the module template
|
||
installed in a modules directory outside the repository are applied, the module writes a row, and
|
||
its directory is removed. Planning then reports the module's history as retained, and the withheld
|
||
Core migration applies with the module absent. The module's table and row survive. Reinstalled, the
|
||
module's history validates strictly again with nothing pending, its row is intact, and a reinstalled
|
||
package whose applied migration changed is refused. A history row for an absent module is retained
|
||
and reported, never fatal, while an unknown row belonging to an installed module is still rejected
|
||
without repair.
|
||
|
||
A dedicated set of scenarios exercises the committed Core identity migration against real
|
||
PostgreSQL: read-only planning on a fresh database, applying it and confirming exactly the four
|
||
expected `ekklesia_core_*` tables and their indexes exist, a genuine no-op on a second apply, Core
|
||
executing before a fixture module migration in one batch, previously applied module-only history
|
||
staying byte-for-byte unchanged once Core is introduced, and a later module migration's failure
|
||
rolling back cleanly without damaging the already-committed Core identity migration or leaking the
|
||
advisory lock.
|
||
|
||
A second dedicated suite (`apps/api/tests/api-http.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) starts the real `apps/api` HTTP server against a freshly
|
||
migrated disposable database and exercises it with real HTTP requests: public sign-up (with no
|
||
automatic session, per `autoSignIn: false`), explicit sign-in producing a host-only
|
||
`HttpOnly`/`SameSite=Lax`/secure session cookie and fixed `{"ok":true}` body containing no token,
|
||
`/api/v1/principal` for both the authenticated and anonymous shapes, sign-out actually invalidating
|
||
the session, a malformed or missing cookie
|
||
resolving as anonymous, duplicate sign-up creating no second identity row, two distinct accounts
|
||
producing identical principal shapes with no elevation and no owner/role/permission data at the
|
||
database level, no plaintext password ever stored, the exact 8–128 password-length boundary (7
|
||
rejected, 8 accepted, 128 accepted, 129 rejected), CORS never returning an allow-origin header for
|
||
an untrusted origin, Better Auth's handler being unreachable outside `/api/auth/*`, a database
|
||
outage during principal resolution producing a sanitized `503` rather than a false anonymous `200`,
|
||
starting the API against an unmigrated database creating no tables at all, and a clean listener/pool
|
||
shutdown.
|
||
|
||
The browser-authentication chain in that live suite also creates a development identity, invokes
|
||
Core's real single-use platform-owner bootstrap for that existing id, signs in through the real HTTP
|
||
transport, presents the returned cookie to `/api/v1/principal`, and reaches protected Dashboard
|
||
navigation with the owner authority. It proves composition across identity, authorization, and the
|
||
Dashboard resource; because production signup is prohibited, it deliberately does **not** claim to
|
||
prove a production first-user enrollment path.
|
||
|
||
A dedicated group within that suite proves session-store readiness against the real database rather
|
||
than a mock, since that is the only place a broken store looks exactly like an anonymous request: a
|
||
request carrying no cookies and one carrying only an unrelated cookie both resolve as anonymous
|
||
while issuing **no** identity database work at all; a request presenting the configured session
|
||
cookie against a dropped session table, a dropped column, and a revoked `SELECT` privilege each
|
||
produce a sanitized `503 PRINCIPAL_RESOLUTION_FAILED` with no relation name, column name, SQL, or
|
||
driver text in the body or the logs; and a genuinely expired or unknown session against a healthy
|
||
store still resolves as anonymous.
|
||
|
||
The same suite also proves Better Auth's own origin/CSRF protection is production-equivalent under
|
||
the test runner, not just in a plain Node script: a no-cookie first authentication attempt succeeds
|
||
from a trusted browser origin and is rejected (`403`, no permissive CORS header, no leaked detail)
|
||
from an untrusted one; a cookie-bearing state-changing request (sign-out) is rejected the same way
|
||
from an untrusted origin, and the presented session remains valid afterward, proving the rejected
|
||
request never reached sign-out logic. See
|
||
[ADR 0012](../architecture/decisions/0012-identity-origin-csrf-protection-under-test.md) for why
|
||
this previously could not be reproduced reliably under Vitest (Better Auth's own default silently
|
||
skips these checks whenever `isTest()` is true, unless `advanced.disableOriginCheck` is set
|
||
explicitly) and how the fix removes that gap entirely. A further scenario forces a real identity
|
||
database failure and asserts, against actual `console.error`/`warn`/`log` output, that Better
|
||
Auth's own internal error logging — configured explicitly in `core/src/identity/runtime.ts` — never
|
||
emits a raw exception, SQL, connection detail, or the rejected request's origin.
|
||
|
||
A dedicated group in the API suite also drives Pass 3's explicit-grant invariants through a real
|
||
authorized grants-management session: a malformed non-UUID revoke returning a sanitized `400` with
|
||
no database work, a canonical unknown UUID returning `404`, a scoped create body returning `400`
|
||
with nothing written, normalized-impossible/non-canonical/past expirations returning `400`, an
|
||
equivalent creation returning a sanitized `409` that names no existing grant, a bounded paginated
|
||
list traversed across three pages with relationship grants absent from every one, and one HTTP
|
||
revocation clearing every unrevoked legacy equivalent with a per-row audit entry.
|
||
|
||
A third dedicated suite (`apps/api/tests/module-database.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) proves `context.services.database` against a real,
|
||
installed test module (`core/tests/fixtures/module-database-integration/db-facade/` — never the
|
||
repository's live `modules/` directory, and invisible to normal discovery/Doctor/`modules:validate`
|
||
unless a test explicitly points discovery at it): the module's own table does not exist until its
|
||
migration is explicitly applied, and constructing the pool or facade never creates it; `start`,
|
||
`stop`, and an authorized HTTP handler all receive the identical `ModuleRuntimeServices` object,
|
||
proven by reference equality; a real authorized mutation persists and a later read observes it,
|
||
gated by the same permission/trusted-Origin enforcement every other module route uses; a real
|
||
committed transaction persists and a real rollback (a deliberate unique-constraint violation)
|
||
leaves no partial mutation; a module deliberately catching a real PostgreSQL failure returns its
|
||
own response untouched, while an uncaught one sanitizes to the existing
|
||
`MODULE_HANDLER_FAILED` boundary with no leaked SQL, error text, or path; and a `stop` hook's write
|
||
completes and is independently verifiable, through the harness's own administrative connection —
|
||
never the API's own pool — before that pool closes. See
|
||
[ADR 0016](../architecture/decisions/0016-module-database-access.md).
|
||
|
||
A fourth dedicated suite (`core/tests/database-tls.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) proves the production PostgreSQL transport policy against a
|
||
real TLS-enabled PostgreSQL rather than against the shape of an options object: a trusted
|
||
certificate whose identity matches the host connects and is confirmed encrypted through
|
||
`pg_stat_ssl`; a certificate that does not chain to the configured CA is refused
|
||
(`UNABLE_TO_VERIFY_LEAF_SIGNATURE`); the same server reached through an address its certificate does
|
||
not cover is refused (`ERR_TLS_CERT_ALTNAME_INVALID`); the identical connection *without* the
|
||
policy's pinned `checkServerIdentity` succeeds, which is the concrete gap the policy closes; and
|
||
`verify-full` cannot be satisfied by a plaintext-only endpoint. Every CA, certificate, and private
|
||
key is generated at test runtime into a temporary directory and deleted afterwards — **no private
|
||
key material is ever committed to this repository** — and the disposable container is removed in
|
||
`afterAll` even when an assertion fails. See
|
||
[ADR 0018](../architecture/decisions/0018-identity-session-resolution-and-database-transport.md).
|
||
|
||
A fifth dedicated suite (`core/tests/module-lifecycle-store.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) proves the module enablement store against real PostgreSQL:
|
||
that migration `0004` really creates `ekklesia_core_module_state` with exactly the four columns and
|
||
the one declared CHECK constraint the schema states — and, load-bearingly, that `enabled` has **no
|
||
column default**, so the platform's default for a newly discovered module is genuinely the absence of
|
||
a row rather than a materialized decision nobody made; that a decision survives a completely new
|
||
connection; that the upsert is idempotent on `module_id` and bumps `updated_at` in place rather than
|
||
inserting a second row; that a malformed identifier is rejected before it reaches PostgreSQL and the
|
||
empty-string case is refused by the database too; and that a record for a module which is no longer
|
||
deployed is retained and simply ignored by resolution, so redeploying restores the decision.
|
||
|
||
The API suite additionally drives the whole lifecycle end to end against a real database and real
|
||
fixture modules: a disabled module's compiled entrypoint is never imported (proven by its
|
||
`register`/`start`/`stop` markers staying empty), its routes return the platform's ordinary `404`,
|
||
its UI artifact and inventory and navigation entries are all absent, and its neighbours run normally;
|
||
a module blocked by a disabled required dependency is explained over HTTP with a structured reason
|
||
naming that dependency and no filesystem path anywhere in the payload; a change made through
|
||
`PUT /api/v1/module-lifecycle/<id>/enabled` reports `restartRequired: true` while the module
|
||
demonstrably keeps serving, and is applied only by the next start; and a disabled module's migrations
|
||
are still discovered, because deployed schema ownership is not runtime enabled state. See
|
||
[ADR 0028](../architecture/decisions/0028-module-operational-lifecycle.md).
|
||
|
||
A sixth dedicated suite (`core/tests/module-configuration-store.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) proves the module configuration store against real
|
||
PostgreSQL: that migration `0005` really creates `ekklesia_core_module_configuration` with the
|
||
composite `(module_id, key)` primary key, the declared CHECK constraints, and **no column that could
|
||
hold secret material**; that a replace is transactional, whole-document, and scoped to one module so
|
||
another module's rows are untouched; that stored values survive a completely new connection; that an
|
||
oversized value is refused by the application *and* by the database; that a store error never carries
|
||
the offending value; and — the security-relevant one — that a module id or key is only ever data in a
|
||
parameterized column, so an injection attempt is a validation error and the table still exists
|
||
afterwards.
|
||
|
||
The API suite additionally proves the configuration boundary end to end: a secret's value never
|
||
appears in any response (including when a row for that key is planted directly in storage), a
|
||
declared-but-undeclared key, a wrong JSON type, a constraint violation, and a `secret` key each fail
|
||
with a distinct sanitized code and store nothing, a reserved property name such as `__proto__` is
|
||
rejected without touching `Object.prototype`, and a module blocked by its own configuration is absent
|
||
from the mountable UI inventory exactly like a disabled one. See
|
||
[ADR 0029](../architecture/decisions/0029-module-configuration-and-secrets.md).
|
||
|
||
A seventh dedicated suite (`core/tests/module-storage.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) proves the object-storage contract against a **real
|
||
S3-compatible server** — the SeaweedFS service the repository's Compose file already pins, brought up
|
||
alongside PostgreSQL by the same harness, so this adds no new CI infrastructure.
|
||
|
||
It exists because mocking would have hidden a real divergence: probing the pinned SeaweedFS 4.28 image
|
||
directly showed that `CreateBucket` on an existing bucket reports `BucketAlreadyExists`, **not** AWS's
|
||
`BucketAlreadyOwnedByYou`. A provider treating only the AWS spelling as success would have failed on
|
||
every second boot against this repository's own development storage, and no unit test would have
|
||
caught it. The suite pins that idempotency, plus a real round trip with content type and unquoted
|
||
ETag preserved, streaming a body without buffering, a Web stream going straight into a `Response`,
|
||
prefix listing and real pagination, a missing object mapping to a structured not-found error carrying
|
||
no endpoint or XML, two modules declaring the same logical bucket staying genuinely isolated on one
|
||
server, the longest bucket name the mapping can produce being accepted, a real connection failure
|
||
mapping to `STORAGE_PROVIDER_UNAVAILABLE`, and a bucket surviving the deletion of every object in it.
|
||
|
||
An eighth suite (`core/tests/module-delivery.integration.ts`, same harness) proves direct browser
|
||
delivery against that same live server, because every claim it makes is a property of the agreement
|
||
between the signer and the provider rather than of code in this repository
|
||
([ADR 0031](../architecture/decisions/0031-direct-object-delivery.md)). It pins: a delivery URL
|
||
returning the object with its content type intact to an ordinary credential-free `fetch`; the same
|
||
object being refused **without** the URL's authorization, so delivery can never be quietly "fixed" by
|
||
making a bucket public; tampered signatures, swapped keys, and swapped buckets all rejected; expiry
|
||
actually enforced; a `Range` request answered directly with `206` and a correct `Content-Range`;
|
||
repeated calls inside a window returning a byte-identical URL so browser caches and URL-keyed DOM
|
||
elements hold; `Cache-Control` never outliving the URL; an optional download filename; and two
|
||
modules' identical logical bucket names resolving to different objects.
|
||
|
||
Its most load-bearing tests concern the **browser delivery origin**. They sign for a public origin and
|
||
replay the request against the real backend through `node:http` with an explicit `Host` header —
|
||
which `fetch` cannot do, since `Host` is a forbidden header name there and it would silently send the
|
||
wrong one and prove nothing. Together they establish that a signature survives a reverse proxy that
|
||
preserves `Host` (even across TLS termination, because the scheme is not signed), that it fails when
|
||
a proxy rewrites `Host`, and that a URL signed for the internal endpoint cannot be rewritten to the
|
||
public one afterwards — which is the entire reason the delivery origin is a signing input rather than
|
||
a post-processing step.
|
||
|
||
An Announcements-shaped fixture in the same file models the real slideshow: it lists its own slides,
|
||
filters images, builds a delivery URL for each, and fetches every one directly. It is written to
|
||
demonstrate what it *cannot* say — its whole world is a `ModuleStorage` and two names from its own
|
||
manifest — and it asserts that the module's durable state holds keys while only the URL carries
|
||
host detail. It also proves that forty URLs cost no provider round trips.
|
||
|
||
A ninth suite (`apps/api/tests/email-verification.integration.ts`, same harness) proves email
|
||
verification and password recovery **end to end**
|
||
([ADR 0038](../architecture/decisions/0038-email-verification-and-password-recovery.md)). Everything
|
||
in it is real: a disposable PostgreSQL, the actual HTTP transport, the actual Better Auth endpoints,
|
||
the actual SMTP provider, and an in-process `smtp-server` accepting the message. The verification and
|
||
reset links are **extracted from the delivered message** and then followed verbatim — the API is
|
||
bound to a port reserved before it is configured, precisely so the emailed URL is the one the test
|
||
resolves rather than one it had to rewrite. A fake mail function would have proved that a callback
|
||
ran and nothing about whether the resulting link works.
|
||
|
||
It pins: an unverified account refused a session with `403 EMAIL_NOT_VERIFIED` and no cookie; the
|
||
full unverified → emailed link → verified → signed in path; a malformed and an expired token
|
||
reported as separate reasons on the host's own landing; verification replay as an idempotent no-op
|
||
that mints nothing; a reset link that changes the password, invalidates the old one, is consumed on
|
||
use, and signs every existing session out; the identity system's own 8–128 policy enforced at reset
|
||
with no reset-specific rule; an unknown address answered byte-identically to a real one; a
|
||
mail-less installation answering identically while recording `MAIL_NOT_CONFIGURED` out of band;
|
||
attacker-supplied `redirectTo`/`callbackURL` values refused, and a *trusted* one still ignored
|
||
because the link is rebuilt from configuration; public signup still refused; and a paired device row
|
||
untouched across a human password reset.
|
||
|
||
The expired-token case is the one place any test constructs a token rather than reading one from a
|
||
delivered message: the verification token is a plain HS256 JWS over the auth secret this suite
|
||
configured, so an already-expired one is forged from that secret with `node:crypto` rather than by
|
||
waiting an hour.
|
||
|
||
**Nothing here reaches the internet and no live credential exists anywhere in it.**
|
||
|
||
`apps/api/tests/initial-owner.integration.ts` additionally proves the trusted-bootstrap exception:
|
||
that the owner is created verified (asserted in the database *and* through a real browser sign-in),
|
||
that neither bootstrap nor sign-in needs any mail configuration, that an adopted identity is marked
|
||
verified too, and that the exception is narrow — an ordinary account created by the identical trusted
|
||
creator stays unverified and is refused.
|
||
|
||
Browser-side, `surfaces/shell/tests/auth-recovery.test.tsx` proves the host-owned screens: that an
|
||
unverified sign-in offers verification rather than a credentials error and never renders the
|
||
Dashboard, that recovery shows one generic completion state whatever the address, that a repeated
|
||
submit sends one email, that a mismatched confirmation is blocked before anything is sent, and that a
|
||
reset token never reaches `localStorage`, `sessionStorage`, the DOM, or the address bar. The landing
|
||
parser is exercised as the pure function it is, including that it names no destination at all.
|
||
|
||
`core/tests/identity-verification-boundaries.test.ts` reads the repository itself for the properties
|
||
no runtime assertion can make: that the identity feature names no SMTP detail, that no Ekklesia
|
||
schema gained a token column and no identity code hashes or compares a token, that nothing
|
||
interpolates a token, link, address, or body into a log line, that a device has no email, password,
|
||
or verification concept, and that no browser source imports Better Auth or Core's mail.
|
||
|
||
Unit suites (`core/tests/module-storage.test.ts`, `core/tests/module-storage-lifecycle.test.ts`) use an
|
||
injected in-memory provider and never touch the network. They prove the parts the protocol cannot:
|
||
binding determinism and collision-safety, scoping and cross-module isolation, key validation, lazy
|
||
provisioning and retry-after-failure, configuration validation and credential redaction, and the
|
||
lifecycle integration including dependency propagation. They also own the delivery bounds, which need
|
||
no server at all: every out-of-range lifetime refused rather than clamped, every download filename
|
||
that could inject a response header refused, and the module-facing surface pinned so a new accessor
|
||
cannot appear unnoticed. See
|
||
[ADR 0030](../architecture/decisions/0030-host-provided-module-object-storage.md) and
|
||
[ADR 0031](../architecture/decisions/0031-direct-object-delivery.md).
|
||
|
||
Phase 7A pass 5 adds realtime evidence at every boundary
|
||
([ADR 0032](../architecture/decisions/0032-native-module-realtime-transport.md)). Contract and Core
|
||
unit suites prove channel grammar, module/permission scoping, staged registration, mandatory initial
|
||
snapshots, atomic snapshot-plus-subscribe, sequence ordering, payload/container limits, latest-state
|
||
coalescing, subscriber limits, exact shutdown behavior, per-module publisher deactivation, sanitized
|
||
errors, and that modules declaring no realtime continue unchanged. Browser SDK/shell suites prove
|
||
that the public contract has no server dependency or transport vocabulary, descriptors and local
|
||
channels are strictly contained, credentials are host-owned, reconnect requires a fresh snapshot,
|
||
terminal EventSource failure cleans up listeners, and explicit unmount is idempotent.
|
||
|
||
`apps/api/tests/module-realtime-routes.test.ts` exercises permission-before-allocation, the
|
||
snapshot/update wire envelope, lack of replay ids, stream headers, comment keepalives, forced
|
||
five-minute reauthorization through a short test-only override, shutdown closure, sanitized
|
||
subscriber failures, and the explicit `HEAD` refusal that prevents an infinite discarded body from
|
||
retaining a subscriber. `apps/api/tests/module-realtime-live.test.ts` binds an actual Node
|
||
HTTP server on an OS-assigned loopback port and drives an Announcements-shaped module through real
|
||
discovery, registration, startup, two live SSE clients, fanout, disconnect cleanup, reconnect with a
|
||
fresh latest snapshot, and shutdown. Every bind, fetch, stream read, cancellation, cleanup poll, and
|
||
listener close is bounded; the test also pins an opaque signed delivery URL byte-for-byte. It uses no
|
||
PostgreSQL or object-store double because neither belongs in realtime transport.
|
||
|
||
The live PostgreSQL API suite adds lifecycle composition evidence: an enabled module's realtime route
|
||
is live while disabled and dependency-blocked modules have no route, just like their HTTP/UI
|
||
surfaces. Its real SSE fetch/read/cancel path is bounded and migrations remain independent of runtime
|
||
enablement. During realtime development, run the focused live Node suite repeatedly as well as
|
||
`pnpm verify` and `pnpm test:migrations:integration`; consecutive runs are specifically intended to
|
||
catch retained timers, listeners, subscribers, or loopback servers.
|
||
|
||
Phase 7A pass 6 adds display evidence across the same boundaries
|
||
([ADR 0033](../architecture/decisions/0033-display-kiosk-host-contract.md)). Contract tests prove the
|
||
manifest v6 hard cutover, optional role combinations, module-owned non-public display permission,
|
||
and collision rejection. Discovery tests prove inert compiled-artifact validation, containment,
|
||
symlink collision detection, deterministic diagnostics, and that modules without a display remain
|
||
valid. Builder tests prove the exact `src/ui/index.tsx` / `src/display/index.tsx` sources,
|
||
deterministic role order, preflight/compile atomicity, no runtime-entrypoint overwrite, and identical
|
||
browser/CSS boundaries.
|
||
|
||
Core/API tests prove independently frozen primary and display publication records, exact anonymous
|
||
artifact routing, permission-gated stable page and descriptor routes, eligible/running lifecycle
|
||
filtering, sanitized failures, and startup-time frozen host assets. Surface tests prove principal-
|
||
first descriptor bootstrap, the role-neutral host context, full-viewport mounting without Dashboard
|
||
chrome or static module imports, late-import cleanup, manual retry, realtime reuse, and unmount.
|
||
The canonical template builds and mounts both artifacts through `parseModuleUiExport`; contract
|
||
fixtures also retain no-display and backend-only cases. These are code-only tests: display hosting
|
||
adds no PostgreSQL, object-store, or Docker requirement beyond the existing suites.
|
||
|
||
A further suite (`apps/api/tests/initial-owner.integration.ts`, same harness) covers the initial
|
||
platform-owner bootstrap ([ADR 0035](../architecture/decisions/0035-initial-platform-owner-bootstrap.md)).
|
||
It deliberately does not stop at asserting rows: a fresh installation is bootstrapped and the
|
||
resulting account is then signed in through the exact browser-facing login transport, its principal
|
||
read from `GET /api/v1/principal`, and used to open a platform-owner-protected resource that stays
|
||
closed to an anonymous caller. Anything less would prove a user was written, not that an operator can
|
||
reach their dashboard.
|
||
|
||
Every scenario runs with `publicSignupEnabled: false` — the production posture — so nothing can
|
||
accidentally depend on the signup route a real deployment does not have, and one scenario asserts
|
||
that a freshly bootstrapped deployment still refuses `POST /api/auth/sign-up/email`. The suite also
|
||
pins the guard (a second run creates no owner, changes no password, and leaves no stray identity),
|
||
the 8–128 password bound and email validation coming from the identity system rather than a
|
||
CLI-specific policy, adoption of an existing identity without touching its password, refusal of a
|
||
credential-less row *without* consuming the single-use sentinel, a partial failure that stays
|
||
recoverable by rerunning, and five racing invocations yielding exactly one owner, one sentinel, and
|
||
one user row.
|
||
|
||
The command itself is exercised as a real child process — success and failure exit codes, the
|
||
password arriving on standard input and never in `argv`, `--password` refused, a missing
|
||
`DATABASE_URL`, and an unmigrated database refused rather than silently migrated — with assertions
|
||
that neither stdout nor stderr contains the password or the connection string.
|
||
|
||
A further suite (`apps/api/tests/device-pairing.integration.ts`, same harness) proves display device
|
||
pairing end to end against real PostgreSQL and the real HTTP transport
|
||
([ADR 0036](../architecture/decisions/0036-managed-device-identity-and-pairing.md)). None of it is
|
||
mocked: a screen with no identity is refused the descriptor but served the page it needs, asks to be
|
||
paired, and receives a code; a real platform owner — created by the Pass 1B bootstrap and signed in
|
||
through the real login transport — looks the code up and approves it by name; the screen collects its
|
||
credential from a real `Set-Cookie` and then opens the real display descriptor and a real realtime
|
||
stream; and the owner revokes it, after which both fail while the device still holds its cookie.
|
||
|
||
The suite also pins the properties that make the design safe rather than merely working: that the
|
||
raw credential appears nowhere in the database or in any response body, that the permission is taken
|
||
from the display declaration and not from a request that tried to name `core.devices.manage`, that
|
||
someone who knows only the code (an onlooker reading a television) is issued nothing, that anonymous
|
||
callers, a paired device, and a cross-origin administrator are all refused approval, that one code
|
||
cannot be approved twice or claimed into two credentials under a race, that re-pairing produces a
|
||
fresh identity while the old credential stays dead, and that a browser holding both a human session
|
||
and a device cookie remains a person.
|
||
|
||
Device authority itself is proven without a database in `core/tests/authorization-device.test.ts`,
|
||
which is mostly a list of things a device must *not* acquire: authority from being a principal, from
|
||
an `authenticated` baseline, from platform ownership, or from a user grant. Its ports are wired to
|
||
answer "yes" to every human question, so any accidental fall-through into a human path would flip
|
||
those assertions.
|
||
|
||
A further suite (`core/tests/module-database-abi.integration.ts`, same harness) proves Core's Module
|
||
Database ABI v1 executor against real PostgreSQL
|
||
([ADR 0050](../architecture/decisions/0050-module-database-abi-v1.md)). Nothing it asserts is mocked,
|
||
because every claim is a property of PostgreSQL or of the protocol:
|
||
- one statement per `query`, enforced by PostgreSQL's extended protocol rather than by parsing, so
|
||
semicolons inside literals, identifiers, and comments are never mistaken for a second statement;
|
||
- every allowed parameter category bound exactly, and every refused one refused before PostgreSQL
|
||
sees it;
|
||
- the decoding table for each type the ADR names;
|
||
- PostgreSQL's SQLSTATE preserved while its message, detail, constraint, and values are hidden;
|
||
- commit, rollback, isolation levels, read-only, no retries, and an aborted transaction never being
|
||
committed;
|
||
- callback-scoped savepoints;
|
||
- a forgotten `await` rolling back instead of committing, while a rejected callback (one half of a
|
||
failed `Promise.all`) keeps its own error;
|
||
- transaction-control SQL sent through `query` detected only after PostgreSQL has run it: Core
|
||
cannot undo a module's own `COMMIT`, so the suite pins what it does do: it refuses everything
|
||
after, never commits, and destroys the connection;
|
||
- duplicate object-mode columns found only after a mutation ran, unless it ran in a transaction;
|
||
- revoked executors reaching nothing.
|
||
|
||
The suite's database is created with deliberately hostile defaults: a non-UTC time zone, a non-ISO
|
||
date style, `sql_standard` intervals, `escape` bytea output, and reduced float digits. So every
|
||
decoding assertion also proves the executor pins its own session settings. The pool-safety cases use
|
||
a one-connection pool, which proves the next borrower gets a connection with exactly the database's
|
||
defaults after every failure class: no transaction, setting, temporary table, listener, or advisory
|
||
lock carries over. The same is true after a connection the server terminated. Parameter validation,
|
||
decoding from text, option validation, error sanitization, and scope bookkeeping are also unit-tested
|
||
without a database in `core/tests/module-database-abi.test.ts`.
|
||
|
||
Outbound email is proven against a **real SMTP server** rather than a mock
|
||
(`core/tests/mail-smtp.integration.test.ts`, [ADR 0037](../architecture/decisions/0037-host-provided-outbound-email.md)).
|
||
`smtp-server` — the reference implementation from the same project as the client — runs in-process,
|
||
so this needs no container and adds no CI infrastructure. **No test sends real email, and no live
|
||
credential exists in this repository or in CI.**
|
||
|
||
It pins the things only a server can settle: a delivered message whose envelope sender and `From:`
|
||
header are the *host's* configured address rather than anything a caller chose; both bodies and a
|
||
reply-to arriving intact; authentication succeeding and its refusal mapping to a sanitized
|
||
`MAIL_REJECTED` that echoes neither the password nor the server's response text; a mailbox refused at
|
||
`RCPT TO` mapping to `MAIL_RECIPIENT_INVALID` rather than a generic failure; a refused connection
|
||
mapping to `MAIL_PROVIDER_UNAVAILABLE`; a server that accepts a connection and never greets hitting
|
||
its deadline instead of hanging; a `starttls` transport refusing to continue when the upgrade is not
|
||
offered; `verify()` connecting without sending anything; and one `send` producing exactly one
|
||
message.
|
||
|
||
`core/tests/mail-contract.test.ts` covers the boundary with no network at all, and is mostly
|
||
negative — the contract's job is to make an impossible message impossible to construct. It also pins
|
||
that a validated message has no `from` key, which is the structural reason no feature can spoof a
|
||
sender. `core/tests/mail-boundaries.test.ts` asserts the properties no runtime test can: the SMTP
|
||
dependency exists in Core alone, no browser-facing package imports it, module SDKs have no way to
|
||
send, no HTTP route references the mail module, Core's mail code names no provider hostname and
|
||
never mentions disabling certificate verification, and `.env.example` documents the variables with
|
||
no password value.
|
||
|
||
### Which kind of timing proof lives where
|
||
|
||
Pass 4 introduced several deadlines, and their tests deliberately differ in kind:
|
||
|
||
- **Unit deadline tests** (`core/tests/module-runtime.test.ts`) drive module stop and rollback
|
||
hooks through a short, test-only `hookTimeoutMs` override with manually controlled
|
||
never-settling promises — no real waiting, no sleeps, and an explicit `unhandledRejection` check
|
||
proving a hook that rejects after its deadline stays handled.
|
||
- **Real raw-HTTP force-close** (`apps/api/tests/api-http.integration.ts`) binds the real listener,
|
||
opens a real socket, writes a deliberately incomplete request, and proves shutdown force-closes
|
||
it at a short `drainTimeoutMsOverride` and still reaches pool closure.
|
||
- **Real TCP connection timeout** (`core/tests/postgres-migration.integration.ts`) binds a loopback
|
||
blackhole listener that accepts the socket and never completes PostgreSQL startup, proving the
|
||
configured connection deadline; the listener and every accepted socket close in `finally`.
|
||
- **Real PostgreSQL operation timeout and SQL batching** (same file) use a bounded server-side
|
||
delay only because the deadline itself is what is under test, and disposable databases to prove
|
||
that marker bytes inside literals, dollar-quoted bodies, comments, and quoted identifiers execute
|
||
intact, and that a failure rolls the whole batch back.
|
||
|
||
Every one of these uses a short **test-only override**; the production defaults (10 s per module
|
||
hook, 30 s HTTP drain, 10 s migration connection/query/statement) are asserted separately as
|
||
constants and are never waited out.
|
||
|
||
CI runs the same command in the dedicated `migration-smoke` job. After this change is merged, add
|
||
the following pull-request status context to `main` branch protection:
|
||
|
||
```text
|
||
CI / migration-smoke (pull_request)
|
||
```
|
||
|
||
## Module packages
|
||
|
||
Ekklesia Module Package v1 (ADR 0049) is proven at three levels, all in `pnpm verify`:
|
||
|
||
- `packages/contracts/tests/module-package.test.ts`: the `ekklesia.package.json` schema, target
|
||
names, and the generated runtime `package.json`.
|
||
- `core/tests/module-package-verification.test.ts`: the verifier against hand-built archives,
|
||
including what the packager never produces. A valid package verifies and extracts with normalized
|
||
modes. A wrong digest, a corrupted or missing or unlisted file, disagreeing identities, an unknown
|
||
format, unknown transitional metadata, an incompatible target, Node.js range, SDK, or database
|
||
runtime, a missing entrypoint, a backend that imports something the package lacks, absolute and
|
||
`..` paths, symbolic and hard links, device nodes and FIFOs, setuid and setgid entries, a second
|
||
top-level directory, and a duplicate entry each fail with their own sanitized code. The backend
|
||
load check runs in a real child process.
|
||
- `packages/module-pack/tests/`: the packager against generated fixture modules (payload contents,
|
||
generated `package.json`, metadata, bundled backend with only built-in imports, refusal of native
|
||
and platform-specific dependencies and of unknown or textless licences, notices, byte-identical
|
||
archives, normalized tar and gzip headers), and end to end against the built module template:
|
||
packed twice to identical bytes without changing the workspace's install state, verified by Core's
|
||
verifier, extracted elsewhere, and loaded with Node's own resolution hooks proving nothing outside
|
||
the package is resolved. Turbo builds the template and Core first, and runs the template's own
|
||
tests (which rebuild its UI in place) before these.
|
||
|
||
## The generic platform image and the reference deployment
|
||
|
||
These need Docker and build images locally; nothing is pushed or published.
|
||
|
||
- `bash scripts/check-image.sh` builds the production image and proves it is the one generic
|
||
platform artifact ADR 0048 requires. It checks that the image has:
|
||
- no `/opt/ekklesia/modules`;
|
||
- no module template;
|
||
- no `ekklesia.module.json` outside installed dependencies;
|
||
- an empty, root-owned `/var/lib/ekklesia/modules`;
|
||
- `EKKLESIA_MODULES_DIR` set, in the image configuration and in a running process.
|
||
|
||
It then builds again from the same checkout with a populated `modules/` directory, and requires
|
||
the same checks and an identical runtime file system. CI's `image-build` job runs it.
|
||
- `pnpm deploy:smoke` (`scripts/smoke-deploy.sh`) runs the reference deployment end to end on the
|
||
generic image with empty module storage. With `EKKLESIA_SMOKE_TOPOLOGY=split`, the same image also
|
||
runs the installed-module lifecycle:
|
||
- the module template is packaged by the canonical packager (Module Package v1, ADR 0049),
|
||
verified by the image's own verifier, and installed into module storage;
|
||
- its migration is planned and explicitly applied, and the API restarted;
|
||
- a real browser mounts it;
|
||
- it is removed and the API restarted, and Core still runs, migrations still plan and apply, and
|
||
its data survives;
|
||
- the exact same package is verified and installed again, and the module returns with its data.
|
||
|
||
See [deploy/README.md](../../deploy/README.md#testing-this-deployment).
|
||
|
||
## Safe troubleshooting
|
||
|
||
Confirm the prerequisites without printing configuration:
|
||
|
||
```bash
|
||
docker info
|
||
docker compose version
|
||
```
|
||
|
||
Then rerun `pnpm test:migrations:integration`. The harness emits fixed failure messages and does
|
||
not dump PostgreSQL logs, SQL, container inspection output, or the generated database URL. Do not
|
||
enable query logging, print the test environment, run `docker compose config` without redaction,
|
||
or paste Docker inspection output into an issue. If the Docker daemon itself stopped during
|
||
cleanup, restore it before rerunning so the harness can create a fresh uniquely named project.
|