ekklesiaandClaude Opus 5
7729c99e24
Ekklesia had no notion of a module being on or off: the accepted module set was exactly "whatever discovery found on disk", and `ModuleLifecycleStage` named nine states nothing produced or consumed. This replaces that vocabulary with two axes that real code produces — persistent, administrator-owned configuration (`enabled`/`disabled`) and per-boot runtime observation (`running`/`disabled`/`blocked`) — and wires them through startup. - One Core table (`ekklesia_core_module_state`, migration 0004) holding only `module_id` and `enabled`. A module with no record is enabled by default, so deploying a module is still sufficient to run it and discovery never needs an initialization write. A record for an undeployed module is retained and ignored, so redeploying restores the decision. - `resolveModuleEligibility` is pure: a disabled module transitively blocks every module that *requires* it (module and capability edges alike) with a structured reason naming the provider; optional dependencies never block; and a missing, incompatible, or cyclic dependency stays discovery's concern. - `startApi` starts only the eligible set and builds the UI publication registry from that same list, so routes, artifacts, browser inventory, and Dashboard navigation all follow one authoritative set. A disabled module is never imported, so there is no per-request enablement check anywhere. - Startup stays all-or-nothing, deliberately; there is consequently no `failed` runtime state. `pnpm modules:enable`/`modules:disable` is the out-of-band recovery path for a module that breaks startup. - Changes take effect at the next start, and the read model reports configuration, runtime, and `restartRequired` separately rather than claiming a module stopped when it has not. - Migration participation and permission composition still span the deployed and discovered inventories respectively, unchanged. - New restricted platform permission `core.modules.lifecycle.manage` gates both reading and changing enablement, at a sibling prefix (`/api/v1/module-lifecycle`) that cannot collide with the module route namespace. - `pnpm doctor` gains a `modules.lifecycle` check so "why is this module unavailable?" is answerable without reading logs. See ADR 0028. Co-Authored-By: Claude Opus 5 <[email protected]>
221 lines
16 KiB
Markdown
221 lines
16 KiB
Markdown
# Testing
|
||
|
||
Ekklesia separates fast code-level evidence from live infrastructure evidence.
|
||
|
||
## Unit and repository verification
|
||
|
||
```bash
|
||
pnpm test
|
||
pnpm verify
|
||
```
|
||
|
||
These commands exercise package tests, repository boundaries, type checking, offline Core and
|
||
module migration discovery, identity schema drift detection, formatting, linting, and builds.
|
||
Migration-runner unit tests use a narrow database/session port to prove deterministic planning,
|
||
approval, transaction orchestration, cleanup, and sanitized errors. They do not claim to prove
|
||
PostgreSQL behavior. Identity unit tests cover branded opaque-ID contracts, principal shapes,
|
||
configuration validation (secret rules, canonical-origin-only base URL and trusted origins, and the
|
||
full loopback/unspecified classifier production rejects — `.localhost`, all of `127.0.0.0/8`,
|
||
`0.0.0.0`, `::1` compressed and expanded, `::`, and IPv4-mapped forms), bounded/validated
|
||
connection and operation (`query_timeout`/`statement_timeout`) database timeouts, the Core identity
|
||
runtime (explicit construction, exact schema mapping, only email/password and rate limiting
|
||
enabled, an explicit `advanced.disableCSRFCheck`/`disableOriginCheck: false` pair, an explicit
|
||
error-only sanitized logger that strips Better Auth's own message and arguments regardless of what
|
||
they contain, pinned 8–128 password-length bounds, sanitized failures, exact whole-name
|
||
session-cookie detection that never fires on a prefixed, suffixed, or differently-cased lookalike,
|
||
and that a broken session store during principal resolution is never silently reported as anonymous
|
||
while an unrelated cookie performs no identity database work at all), the PostgreSQL transport
|
||
policy (`core/tests/database-transport.test.ts`: mode validation including every loose or aliased
|
||
spelling, production's plaintext refusal with no loopback exception, connection-string
|
||
TLS-parameter rejection, and that no configuration path yields `rejectUnauthorized: false` or leaks
|
||
CA material into an error), and Core migration discovery (deterministic ordering, the reserved
|
||
`core` identity, and collision rejection) — all without a live database. `apps/api`'s own unit tests cover its pure Hono app factory (route shapes, module
|
||
route mounting/enforcement/CORS/response sanitization, CORS allow/deny/preflight, deterministic 404
|
||
and sanitized error responses), its runtime configuration, its idempotent shutdown lifecycle
|
||
(concurrent callers share one server-close-then-module-runtime-stop-then-pool-close promise; a
|
||
module-stop failure never prevents pool cleanup; a startup failure after the pool was already
|
||
created still closes it, stopping the module runtime first if it had already started;
|
||
`SIGINT`/`SIGTERM` wiring is exercised through an injectable exit function rather than the real
|
||
`process.exit`), and that importing any of its modules — including the executable server module
|
||
without calling its exported `main` — never reads the environment, connects to PostgreSQL, imports
|
||
a module entrypoint, or starts a server. Real, working (non-throwing) module fixtures prove
|
||
production module-runtime startup/shutdown ordering and cleanup against a mocked pool/Better
|
||
Auth/server — see [Authorization](authorization.md#tests).
|
||
|
||
`pnpm verify` is intentionally Docker-free. It never plans or applies migrations against a live
|
||
database.
|
||
|
||
## Live PostgreSQL migration evidence
|
||
|
||
Docker and the Docker Compose plugin are required for the live suite:
|
||
|
||
```bash
|
||
pnpm test:migrations:integration
|
||
```
|
||
|
||
The command first builds the workspace through the repository's Turbo build graph, then starts the
|
||
repository's pinned PostgreSQL image in a unique Compose project, waits for bounded health
|
||
readiness, runs the production PostgreSQL adapter and migration services, and removes the
|
||
container, network, and volume. A build failure occurs before Docker provisioning. Each scenario
|
||
also receives a safely named disposable database. Direct host runs use a Docker-assigned random
|
||
port bound only to `127.0.0.1`; a containerized Gitea job joins the isolated Compose network and
|
||
uses the PostgreSQL service name.
|
||
|
||
The live suite proves fresh read-only planning, initial and incremental application, genuine
|
||
no-op behavior, exact applied checksums and history, edited or corrupt history rejection,
|
||
whole-batch DDL rollback, Drizzle breakpoint execution, real advisory-lock concurrency and
|
||
timeouts, lock release after failure, exact destructive approvals, and public-error sanitization.
|
||
It does not make migrations automatic during Core startup.
|
||
|
||
A dedicated set of scenarios exercises the committed Core identity migration against real
|
||
PostgreSQL: read-only planning on a fresh database, applying it and confirming exactly the four
|
||
expected `ekklesia_core_*` tables and their indexes exist, a genuine no-op on a second apply, Core
|
||
executing before a fixture module migration in one batch, previously applied module-only history
|
||
staying byte-for-byte unchanged once Core is introduced, and a later module migration's failure
|
||
rolling back cleanly without damaging the already-committed Core identity migration or leaking the
|
||
advisory lock.
|
||
|
||
A second dedicated suite (`apps/api/tests/api-http.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) starts the real `apps/api` HTTP server against a freshly
|
||
migrated disposable database and exercises it with real HTTP requests: public sign-up (with no
|
||
automatic session, per `autoSignIn: false`), explicit sign-in producing a host-only
|
||
`HttpOnly`/`SameSite=Lax`/secure session cookie, `/api/v1/principal` for both the authenticated and
|
||
anonymous shapes, sign-out actually invalidating the session, a malformed or missing cookie
|
||
resolving as anonymous, duplicate sign-up creating no second identity row, two distinct accounts
|
||
producing identical principal shapes with no elevation and no owner/role/permission data at the
|
||
database level, no plaintext password ever stored, the exact 8–128 password-length boundary (7
|
||
rejected, 8 accepted, 128 accepted, 129 rejected), CORS never returning an allow-origin header for
|
||
an untrusted origin, Better Auth's handler being unreachable outside `/api/auth/*`, a database
|
||
outage during principal resolution producing a sanitized `503` rather than a false anonymous `200`,
|
||
starting the API against an unmigrated database creating no tables at all, and a clean listener/pool
|
||
shutdown.
|
||
|
||
A dedicated group within that suite proves session-store readiness against the real database rather
|
||
than a mock, since that is the only place a broken store looks exactly like an anonymous request: a
|
||
request carrying no cookies and one carrying only an unrelated cookie both resolve as anonymous
|
||
while issuing **no** identity database work at all; a request presenting the configured session
|
||
cookie against a dropped session table, a dropped column, and a revoked `SELECT` privilege each
|
||
produce a sanitized `503 PRINCIPAL_RESOLUTION_FAILED` with no relation name, column name, SQL, or
|
||
driver text in the body or the logs; and a genuinely expired or unknown session against a healthy
|
||
store still resolves as anonymous.
|
||
|
||
The same suite also proves Better Auth's own origin/CSRF protection is production-equivalent under
|
||
the test runner, not just in a plain Node script: a no-cookie first authentication attempt succeeds
|
||
from a trusted browser origin and is rejected (`403`, no permissive CORS header, no leaked detail)
|
||
from an untrusted one; a cookie-bearing state-changing request (sign-out) is rejected the same way
|
||
from an untrusted origin, and the presented session remains valid afterward, proving the rejected
|
||
request never reached sign-out logic. See
|
||
[ADR 0012](../architecture/decisions/0012-identity-origin-csrf-protection-under-test.md) for why
|
||
this previously could not be reproduced reliably under Vitest (Better Auth's own default silently
|
||
skips these checks whenever `isTest()` is true, unless `advanced.disableOriginCheck` is set
|
||
explicitly) and how the fix removes that gap entirely. A further scenario forces a real identity
|
||
database failure and asserts, against actual `console.error`/`warn`/`log` output, that Better
|
||
Auth's own internal error logging — configured explicitly in `core/src/identity/runtime.ts` — never
|
||
emits a raw exception, SQL, connection detail, or the rejected request's origin.
|
||
|
||
A dedicated group in the API suite also drives Pass 3's explicit-grant invariants through a real
|
||
authorized grants-management session: a malformed non-UUID revoke returning a sanitized `400` with
|
||
no database work, a canonical unknown UUID returning `404`, a scoped create body returning `400`
|
||
with nothing written, normalized-impossible/non-canonical/past expirations returning `400`, an
|
||
equivalent creation returning a sanitized `409` that names no existing grant, a bounded paginated
|
||
list traversed across three pages with relationship grants absent from every one, and one HTTP
|
||
revocation clearing every unrevoked legacy equivalent with a per-row audit entry.
|
||
|
||
A third dedicated suite (`apps/api/tests/module-database.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) proves `context.services.database` against a real,
|
||
installed test module (`core/tests/fixtures/module-database-integration/db-facade/` — never the
|
||
repository's live `modules/` directory, and invisible to normal discovery/Doctor/`modules:validate`
|
||
unless a test explicitly points discovery at it): the module's own table does not exist until its
|
||
migration is explicitly applied, and constructing the pool or facade never creates it; `start`,
|
||
`stop`, and an authorized HTTP handler all receive the identical `ModuleRuntimeServices` object,
|
||
proven by reference equality; a real authorized mutation persists and a later read observes it,
|
||
gated by the same permission/trusted-Origin enforcement every other module route uses; a real
|
||
committed transaction persists and a real rollback (a deliberate unique-constraint violation)
|
||
leaves no partial mutation; a module deliberately catching a real PostgreSQL failure returns its
|
||
own response untouched, while an uncaught one sanitizes to the existing
|
||
`MODULE_HANDLER_FAILED` boundary with no leaked SQL, error text, or path; and a `stop` hook's write
|
||
completes and is independently verifiable, through the harness's own administrative connection —
|
||
never the API's own pool — before that pool closes. See
|
||
[ADR 0016](../architecture/decisions/0016-module-database-access.md).
|
||
|
||
A fourth dedicated suite (`core/tests/database-tls.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) proves the production PostgreSQL transport policy against a
|
||
real TLS-enabled PostgreSQL rather than against the shape of an options object: a trusted
|
||
certificate whose identity matches the host connects and is confirmed encrypted through
|
||
`pg_stat_ssl`; a certificate that does not chain to the configured CA is refused
|
||
(`UNABLE_TO_VERIFY_LEAF_SIGNATURE`); the same server reached through an address its certificate does
|
||
not cover is refused (`ERR_TLS_CERT_ALTNAME_INVALID`); the identical connection *without* the
|
||
policy's pinned `checkServerIdentity` succeeds, which is the concrete gap the policy closes; and
|
||
`verify-full` cannot be satisfied by a plaintext-only endpoint. Every CA, certificate, and private
|
||
key is generated at test runtime into a temporary directory and deleted afterwards — **no private
|
||
key material is ever committed to this repository** — and the disposable container is removed in
|
||
`afterAll` even when an assertion fails. See
|
||
[ADR 0018](../architecture/decisions/0018-identity-session-resolution-and-database-transport.md).
|
||
|
||
A fifth dedicated suite (`core/tests/module-lifecycle-store.integration.ts`, included in the same
|
||
`pnpm test:migrations:integration` run) proves the module enablement store against real PostgreSQL:
|
||
that migration `0004` really creates `ekklesia_core_module_state` with exactly the four columns and
|
||
the one declared CHECK constraint the schema states — and, load-bearingly, that `enabled` has **no
|
||
column default**, so the platform's default for a newly discovered module is genuinely the absence of
|
||
a row rather than a materialized decision nobody made; that a decision survives a completely new
|
||
connection; that the upsert is idempotent on `module_id` and bumps `updated_at` in place rather than
|
||
inserting a second row; that a malformed identifier is rejected before it reaches PostgreSQL and the
|
||
empty-string case is refused by the database too; and that a record for a module which is no longer
|
||
deployed is retained and simply ignored by resolution, so redeploying restores the decision.
|
||
|
||
The API suite additionally drives the whole lifecycle end to end against a real database and real
|
||
fixture modules: a disabled module's compiled entrypoint is never imported (proven by its
|
||
`register`/`start`/`stop` markers staying empty), its routes return the platform's ordinary `404`,
|
||
its UI artifact and inventory and navigation entries are all absent, and its neighbours run normally;
|
||
a module blocked by a disabled required dependency is explained over HTTP with a structured reason
|
||
naming that dependency and no filesystem path anywhere in the payload; a change made through
|
||
`PUT /api/v1/module-lifecycle/<id>/enabled` reports `restartRequired: true` while the module
|
||
demonstrably keeps serving, and is applied only by the next start; and a disabled module's migrations
|
||
are still discovered, because deployed schema ownership is not runtime enabled state. See
|
||
[ADR 0028](../architecture/decisions/0028-module-operational-lifecycle.md).
|
||
|
||
### Which kind of timing proof lives where
|
||
|
||
Pass 4 introduced several deadlines, and their tests deliberately differ in kind:
|
||
|
||
- **Unit deadline tests** (`core/tests/module-runtime.test.ts`) drive module stop and rollback
|
||
hooks through a short, test-only `hookTimeoutMs` override with manually controlled
|
||
never-settling promises — no real waiting, no sleeps, and an explicit `unhandledRejection` check
|
||
proving a hook that rejects after its deadline stays handled.
|
||
- **Real raw-HTTP force-close** (`apps/api/tests/api-http.integration.ts`) binds the real listener,
|
||
opens a real socket, writes a deliberately incomplete request, and proves shutdown force-closes
|
||
it at a short `drainTimeoutMsOverride` and still reaches pool closure.
|
||
- **Real TCP connection timeout** (`core/tests/postgres-migration.integration.ts`) binds a loopback
|
||
blackhole listener that accepts the socket and never completes PostgreSQL startup, proving the
|
||
configured connection deadline; the listener and every accepted socket close in `finally`.
|
||
- **Real PostgreSQL operation timeout and SQL batching** (same file) use a bounded server-side
|
||
delay only because the deadline itself is what is under test, and disposable databases to prove
|
||
that marker bytes inside literals, dollar-quoted bodies, comments, and quoted identifiers execute
|
||
intact, and that a failure rolls the whole batch back.
|
||
|
||
Every one of these uses a short **test-only override**; the production defaults (10 s per module
|
||
hook, 30 s HTTP drain, 10 s migration connection/query/statement) are asserted separately as
|
||
constants and are never waited out.
|
||
|
||
CI runs the same command in the dedicated `migration-smoke` job. After this change is merged, add
|
||
the following pull-request status context to `main` branch protection:
|
||
|
||
```text
|
||
CI / migration-smoke (pull_request)
|
||
```
|
||
|
||
## Safe troubleshooting
|
||
|
||
Confirm the prerequisites without printing configuration:
|
||
|
||
```bash
|
||
docker info
|
||
docker compose version
|
||
```
|
||
|
||
Then rerun `pnpm test:migrations:integration`. The harness emits fixed failure messages and does
|
||
not dump PostgreSQL logs, SQL, container inspection output, or the generated database URL. Do not
|
||
enable query logging, print the test environment, run `docker compose config` without redaction,
|
||
or paste Docker inspection output into an issue. If the Docker daemon itself stopped during
|
||
cleanup, restore it before rerunning so the harness can create a fresh uniquely named project.
|